mirrors
/
sqlite
mirror of https://github.com/sqlite/sqlite
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
18,226 Commits 1,474 Branches 166 Tags 514 MiB
Commit Graph

440 Commits

Author SHA1 Message Date
dan 159fd77e0f Fix an fts5 integrity-check problem that affects offsets=0 tables with prefix indexes. 
FossilOrigin-Name: 609a0bc7f34e6dae74ce756aff920f3df78fe828
 2015-12-21 18:45:09 +00:00
dan c58b9eeaaa Fix a problem with prefix queries on fts5 offsets=0 tables. 
FossilOrigin-Name: ad0987d83c252dd8d6a69321893629d7be805c28
 2015-12-18 19:07:14 +00:00
dan b12dc84fbb Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch. 
FossilOrigin-Name: 40b5bbf02a824ca73b33aa4ae1c7d5f65b7cda10
 2015-12-17 20:36:13 +00:00
mistachkin b9becaa268 Fix even more harmless compiler warnings. 
FossilOrigin-Name: 1d0e6aa119da8e15d35508f5d75ffc729979da92
 2015-12-16 23:30:30 +00:00
dan f5d8c58950 Fix the fts5 "prefix=" option to match the documentation (space separated list, multiple prefix= options supported). The undocumented comma-separated format (compatible with fts4) still works. 
FossilOrigin-Name: 11eb8e877e2ba859ef6b44318f286597186dfaf2
 2015-11-25 11:56:24 +00:00
drh 43509768e1 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 09752e51a18ac1b4c9642965e6ee1b6a32de00df
 2015-11-10 12:31:25 +00:00
dan dbbda39453 Have fts5 load its configuration from the xConnect() method is invoked. This ensures that the very first query run uses the correct value of the 'rank' option. 
FossilOrigin-Name: 33e6606f5e497e81119ec491cf2370f60bddafc0
 2015-11-06 12:50:57 +00:00
mistachkin 9030d0ac28 Fix typo in help information for an FTS5 script. 
FossilOrigin-Name: 777ae8007f6ff303b120b25f2dc37d7ef6b6a4f8
 2015-11-05 21:49:19 +00:00
dan d82211db56 Add the 'hashsize' configuration option to fts5, for configuring the amount of memory allocated to the in-memory hash table while writing. 
FossilOrigin-Name: 445480095e6877cce8220b1c095f334bbb04c1c3
 2015-11-05 18:09:16 +00:00
dan 36c33aa633 Have contentless and external content fts5 tables ignore "OR REPLACE" conflict handling. 
FossilOrigin-Name: a85c2a4758c27e8d5d0395751eb3cfd9985ce696
 2015-10-27 20:04:53 +00:00
dan 24b8457911 Fix problems with prefix queries in fts5. 
FossilOrigin-Name: 3b5758c647530bd5c2e68d0ee3e9f58a96347ca4
 2015-10-27 17:48:57 +00:00
mistachkin 16158eeb49 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: aa4e01ea1af327d1f8398ebea1c5bacc46698c3d
 2015-10-21 22:08:36 +00:00
dan 99b25309d0 Remove some branches made unreachable by recent changes from fts5. 
FossilOrigin-Name: ae350bfbfd59f912c469a51bf7d1409f3c8d958a
 2015-10-21 20:56:27 +00:00
dan f2c9995dbe Minor optimization for fts5 queries. 
FossilOrigin-Name: 363b36d50b6f3e3d3e79c538be1167b071a5a710
 2015-10-21 20:07:08 +00:00
mistachkin 6d7734c27d Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 0a903ec26bfbbe9bfd18bc34934e689ffdb01ce5
 2015-10-20 23:27:14 +00:00
dan 219f4d48a2 More optimizations for fts5 prefix queries. 
FossilOrigin-Name: b8fb263ed1b36d085437131795505e53af103d26
 2015-10-20 21:05:53 +00:00
dan 9cc4ae6f2a Fix #line directives added to generated file fts5.c. 
FossilOrigin-Name: 1f5f5804cd394a81b1c0712ce6d1052f228d0a40
 2015-10-20 19:55:35 +00:00
dan 3f9bfa740f Initialize variables in the fts5 integrity-check code to avoid compiler warnings. 
FossilOrigin-Name: e979e2cccac257ad554803e6a1103558b0f159d8
 2015-10-20 15:49:02 +00:00
dan 8789fbb2bd Another optimization for fts5 prefix (and other) queries. 
FossilOrigin-Name: 60a8bde055a960c5b8cb4e231802c75617c942d8
 2015-10-19 20:49:10 +00:00
dan 5e5b454cd8 Another tweak to improve performance of fts5 prefix queries. 
FossilOrigin-Name: 69be427c864dc3de804ca3c5f1c6addcd33cd188
 2015-10-19 17:43:24 +00:00
dan 3c060c2aef Fix a problem in the fts5txt2db.tcl script causing it to fail when creating tables with more than 8 columns. 
FossilOrigin-Name: 68ee426a6432ac63655909ff70cb22bde33cf28c
 2015-10-19 15:59:08 +00:00
dan 0206920e27 Add tests to cover new branches added to the fts5 code by the previous commit. 
FossilOrigin-Name: cf618334a8a05a067e844236e4d461b2e5706da6
 2015-10-17 20:34:49 +00:00
dan 05903e6a87 Handle equality and range constraints on the "term" column of fts5vocab tables. Also have the same module report columns using names instead of indexes. 
FossilOrigin-Name: 6020d96ab47a0bebe41f691a3f44fd55d8b2d2b7
 2015-10-17 19:49:01 +00:00
dan 5f93ccd7fc Tests and minor fixes to improve coverage of FTS5 code. 
FossilOrigin-Name: f4de6d450e143cb1dd5447800d4b178e3db114c8
 2015-10-14 21:08:48 +00:00
mistachkin cdabd7bd50 Fix harmless compiler warnings. 
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
 2015-10-14 20:34:57 +00:00
dan f4f968443b Add the missing 'extern "C" {...}' blocks to fts5.h and sqlite3rbu.h. 
FossilOrigin-Name: 47a2ce97d585e1760ffcf760c0de1753677d5713
 2015-10-14 10:56:34 +00:00
drh cec7dec006 Remove the unused fts5ExprColsetTest() routine. 
FossilOrigin-Name: 9ecafc0c94102dca6da192f400278399464086fb
 2015-10-12 22:31:02 +00:00
dan df705d8185 Further tweaks to improve fts5 prefix query performance. 
FossilOrigin-Name: 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
 2015-10-12 19:12:29 +00:00
drh 72724da95a Remove another instance of strcpy() from FTS5, to mollify OpenBSD. 
FossilOrigin-Name: 35e6248abb4435a8b26d270092b856beff867406
 2015-10-10 15:57:20 +00:00
drh c5971cc859 Omit the use of strcpy() in FTS5 since OpenBSD hates strcpy(). 
FossilOrigin-Name: bc24a5bbfd95df3518611b221de69b73776111bc
 2015-10-10 15:11:49 +00:00
dan 0008ce7cc5 Makefile and fts5 changes so that fts5 works with main.mk. Still does not work with Makefile.in. 
FossilOrigin-Name: 6ebac32d8378e46f44a6eb55aaa30baf4d7dde7d
 2015-10-09 11:09:47 +00:00
drh 45fac88951 Trying to get FTS5 tests to build and run using main.mk. Not yet working. 
FossilOrigin-Name: 206174f9b154c590a3e2e94236db38e111d64bdb
 2015-10-09 01:42:49 +00:00
dan d3b90a2e41 Remove calls to "load_static_extension" from fts5 test scripts. 
FossilOrigin-Name: 53dd7c4c65842c5c9ed3e84edd62e24f2c9be15b
 2015-10-08 21:02:29 +00:00
dan 23fda8242c Further tweaks to main.mk so that fts5 builds as part of testfixture. 
FossilOrigin-Name: 97e11b79a52fb1c57f0afd2dbd6ee66cf00a922d
 2015-10-08 21:00:46 +00:00
dan 56eac26897 Fix further fts5 compilation issues. 
FossilOrigin-Name: 0d7f301fb03c8546a5dbce5d061a1d74e3c86bcf
 2015-10-08 20:40:18 +00:00
dan 351ccb3825 Fix two more warnings in fts5 code. 
FossilOrigin-Name: 5343e601fb6f0f24f4e4d5d442f7b5b074734143
 2015-10-08 20:28:33 +00:00
dan 056d7acf25 Fix a couple of errors in fts5 compilation. This branch still does not build with -Wall -Werror. 
FossilOrigin-Name: e9e76e12812b9b42b1142246b71cea98e26903aa
 2015-10-08 20:11:42 +00:00
drh 50065656f7 First attempt to add json1 and fts5 to the amalgamation. This check-in does 
not compile.

FossilOrigin-Name: d820a1bd1b8dd702d1b3a099af393530cc5a5d51
 2015-10-08 19:29:18 +00:00
dan 98e713bb8d Further optimizations for fts5 prefix queries without a prefix index. 
FossilOrigin-Name: 83dc1ff7fa010715ca7f406a572f4ee444a967d7
 2015-10-07 19:06:21 +00:00
mistachkin 08114efde4 Fix harmless compiler warning in FTS5. 
FossilOrigin-Name: 13adcd038fc20dd1b6f344f79b449b4034f8f8f2
 2015-10-07 17:06:17 +00:00
dan 22c7cca21f Allow the subsitute character (codepoint 26 - 0x1A) to appear in fts5 barewords. 
FossilOrigin-Name: baec1b96cb64a6a6317143a8df841e502afe3914
 2015-10-07 17:01:22 +00:00
dan d46cc33131 Add tests for fts5 phrase queries with column filters. 
FossilOrigin-Name: f20f9f813f00cefcd427e506a09b0b973c782e3f
 2015-10-07 13:24:27 +00:00
dan 92099b0753 Add further tests for fts5 prefix queries. 
FossilOrigin-Name: accdc98b1291f07b802fd23f3ebc7dbc02ba09d3
 2015-10-07 09:02:50 +00:00
dan a2507137f3 Optimizations for fts5 expressions that filter on column. More still to come. 
FossilOrigin-Name: bf1607ac155018573ca40fb58aca62c5fea7e60b
 2015-10-06 20:53:26 +00:00
dan 6f832d3dff Improve performance of prefix queries without a prefix index on fts5 tables. 
FossilOrigin-Name: f2f0184e9e1c9f121ee2ac864cd28c8cd8efecb5
 2015-10-05 19:41:16 +00:00
dan 2c1023df09 Add fts5txt2db.tcl, a tool for creating sample fts4/5 databases from text files. 
FossilOrigin-Name: 44f1ce30d1e446c9ee5f8bd8a62119e00356aa0e
 2015-10-05 11:57:09 +00:00
dan a0f197b842 Update fts5 to support the table function syntax. "... FROM fts5_tbl WHERE fts5_tbl MATCH ?1 AND rank MATCH ?1" can now be written "FROM fts5_tbl(?1, ?2)". 
FossilOrigin-Name: 41d17d9e24303aed6d549ea23c62b93bf2f71691
 2015-10-03 15:38:57 +00:00
dan bca189d5d8 Update fts5 to avoid using a statement journal for UPDATE and DELETE operations that affect at most a single row. 
FossilOrigin-Name: 5c83b9db46d61cfa76a1abed50467e2f02db0eb0
 2015-10-02 20:04:30 +00:00
dan 4ad0971919 Fix an off-by-one error in test function fts5_decode(). 
FossilOrigin-Name: 3a9f076250d9559d8ea94ba44095ba3ddbc5542d
 2015-09-29 12:19:51 +00:00
dan cf346d7a9c Fix a crash in FTS5 caused by specifying NULL as the rhs of a MATCH operator. 
FossilOrigin-Name: e8ed62f82e8acc40b818bf86fafe3d480687514e
 2015-09-17 09:48:01 +00:00
dan ec9e8f004e Fix typo in fts5.h. 
FossilOrigin-Name: 07721c6c24f651d8e281c29227b113d292f6b0de
 2015-09-16 12:31:39 +00:00
dan 6609bab24d Add test cases to cover fts5 integrity-check code. 
FossilOrigin-Name: 1d018c35b9e81982df036f5e62a4a42219b54e02
 2015-09-15 14:39:17 +00:00
dan 128ea696ea Fix a problem with fts5 "ORDER BY rowid DESC" queries and large terms. 
FossilOrigin-Name: b26d8f79c62cb71b4c3ea5df9d0e09558bbbd794
 2015-09-15 11:58:33 +00:00
dan dafeda754b Improve test coverage of fts5_index.c. 
FossilOrigin-Name: c1f76686cee3918b1be785a4071d68cb3afda0ef
 2015-09-14 19:51:05 +00:00
mistachkin 16a93128f8 Fix harmless compiler warnings. 
FossilOrigin-Name: bfc7b84b766860d2e410702ba7c1166d7328309a
 2015-09-11 18:05:01 +00:00
dan bcb04b6128 Fix some compiler warnings in fts5 code. 
FossilOrigin-Name: 0dc436116e55e3fd55eb6085ada71e099069b32d
 2015-09-11 14:15:46 +00:00
dan 204debf3a3 Increment the fts5 version value to indicate that the on-disk format has changed. 
FossilOrigin-Name: 99de5e3613d557728dd196353516bc7cf64a0e6c
 2015-09-10 16:39:38 +00:00
dan 56c8634853 Fix a segfault in fts5 that could occur if the database contents were corrupt. 
FossilOrigin-Name: 4931e37da4d2c26d7afc5432f7f0d534b51a85fa
 2015-09-10 16:19:01 +00:00
dan 40d45f8960 Merge latest changes from trunk. Including fts5_expr.c fixes. 
FossilOrigin-Name: 716e7e747714d6af502f6a87ca8d789bb7ce162a
 2015-09-10 15:52:42 +00:00
dan b97efa0ba3 Update description of on-disk format in fts5_index.c. 
FossilOrigin-Name: 85aac7b8b6731e2f6880b80cfd62d431ea059799
 2015-09-10 15:49:16 +00:00
dan f679d97d18 Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages. 
FossilOrigin-Name: ffe2796ac9244c62325fce4960f26c653321623c
 2015-09-10 10:01:30 +00:00
dan a5aa8e1db2 Change the array of 16-bit offsets at the end of each page to an array of varints. 
FossilOrigin-Name: fab245bea4f283714c17bca22428d5eb4db5935a
 2015-09-10 05:40:17 +00:00
mistachkin f5eac36a64 Fix harmless compiler warning in FTS5. 
FossilOrigin-Name: 86146a731d75eb25279c0e072c0bdda593de905d
 2015-09-09 23:54:46 +00:00
mistachkin 4b03efb695 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 2cdb18778f20baa902e54f218ba26d2bacf0801e
 2015-09-09 17:17:22 +00:00
dan f06bd2c132 Fix a bug in preprocessor macros within fts5_main.c. 
FossilOrigin-Name: 0eb2b9521fad6fa36e6fa374c2bc1f70b5180f7c
 2015-09-09 08:15:06 +00:00
dan fe8e2eba0a Remove the 0x00 terminators from the end of doclists stored on disk. 
FossilOrigin-Name: 00d990061dec3661b0376bd167082942d5563bfe
 2015-09-08 19:55:26 +00:00
dan 60f8139a96 Use macros to make the code in fts5_index.c easier to read. 
FossilOrigin-Name: 67ff5ae81357eb7fa28049bb724a22cb6f52e076
 2015-09-07 08:14:30 +00:00
dan e386a1ba25 Experiment with a different fts5 leaf page format that allows faster seeks. 
FossilOrigin-Name: a1f4c3b543eed84e808f6b901a38179786fffe16
 2015-09-05 19:52:08 +00:00
dan 4dfe98a8c4 Enhance showfts5.tcl so that it can optionally display the number of terms in each segment. 
FossilOrigin-Name: d648ddd93de039820f5abe064c7bc1318cd9d6b1
 2015-09-04 11:13:00 +00:00
dan f8efe64649 Rearrange code in fts5_expr.c so that synonym support does not slow down the common case. 
FossilOrigin-Name: 801882817f1d895aef1426f9a7196bd140b807c3
 2015-09-03 19:48:02 +00:00
dan ea8b93ee20 Add documentation for fts5 synonym support. 
FossilOrigin-Name: 58aa1f435959852df74f1bca8e0bdbc4f47c256a
 2015-09-03 18:05:09 +00:00
dan ea623630e2 Add tests to improve coverage of fts5_varint.c. 
FossilOrigin-Name: 89f24f31a8f7d7cb0a66ee53523881f566dcb035
 2015-09-03 15:37:26 +00:00
dan e1c28d7229 Remove some more code from fts5_index.c by consolidating similar functions. 
FossilOrigin-Name: 59ae30b97b40faa363c55aa2664dead9eaeeddc0
 2015-09-03 14:22:27 +00:00
dan 5d144875bc Remove dead code from fts5_index.c. 
FossilOrigin-Name: 8a0a9b01e74072ee52fe393311ad591208fbbf7c
 2015-09-03 11:17:52 +00:00
dan 9a9e3531f7 Fix the fts5 integrity-check so that it works with columnsize=0 tables. 
FossilOrigin-Name: 11b887b15eaee57ea2577c763e70494f1e251275
 2015-09-03 11:00:50 +00:00
dan 2a6ecb3141 Fix a memory leak in fts5_expr.c. 
FossilOrigin-Name: 399932a181437d84cd57234e76c4b65da1e4ac5f
 2015-09-03 10:27:02 +00:00
dan 9c671b741c Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same. 
FossilOrigin-Name: bdedd838bb3028c586bcc9f643852ce1364adb49
 2015-09-02 19:48:55 +00:00
dan 50ea76e748 Fix an issue with fts5 synonyms and NEAR(...) queries. 
FossilOrigin-Name: f2e590700d8eb30da8ac3a92f6ca92a2bbe9c1fe
 2015-09-02 18:56:01 +00:00
dan bea34fc53a Fix a problem with fts5 synonyms and phrase queries. Also fix an OOM handling bug in fts5. 
FossilOrigin-Name: a4c35fa2c94fe34b376670244fe72303c99868c1
 2015-09-02 17:34:22 +00:00
dan 583611df65 Fix a problem handling OOM conditions within fts5 queries that feature synonyms. 
FossilOrigin-Name: 11fa980897c6c7be218bbd9c4cd8253272d2c300
 2015-09-02 14:17:38 +00:00
dan df998c3d4f Fix a problem with fts5 synonyms and the xQueryPhrase() auxiliary function API. 
FossilOrigin-Name: cf3e45e76d23e10ee06296c3561a341591597a04
 2015-09-02 08:22:41 +00:00
dan d917ad019c Fix a problem that occurs when more than 4 synonyms for a term appear within a single row. 
FossilOrigin-Name: cd359550bdc2bf7be4c294b60130c9fc3f583902
 2015-09-01 18:44:33 +00:00
dan 3faa07ee04 Add tests for fts5 synonyms implemented by adding extra terms to queries. And fixes for the same. 
FossilOrigin-Name: dbcb73802b88f76be17f09f3eb83ffac542de633
 2015-09-01 18:08:36 +00:00
dan cb8b58caf7 Begin changes to allow synonym support by adding multiple terms to a query (an alternative to adding multiple terms to the FTS index). 
FossilOrigin-Name: ad7feaed4cd6b1d6e6376bb82d1f5664ddd083f3
 2015-08-31 20:06:06 +00:00
dan 0cb01f5c2c Add a test for an fts5 tokenizer that supports synonyms by adding multiple entries to the fts index. 
FossilOrigin-Name: 98d07d16cab92f1e7001afbe370df3ec6343fc1f
 2015-08-29 18:46:12 +00:00
dan ee0c0a8de3 Another change to the fts5 tokenizer API. 
FossilOrigin-Name: fc71868496f45f9c7a79ed2bf2d164a7c4718ce1
 2015-08-29 15:44:27 +00:00
dan 57e0add3f9 Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document. 
FossilOrigin-Name: 90b85b42f2b2dd3e939b129b7df2b822a05e243d
 2015-08-28 19:56:47 +00:00
mistachkin 0cbb513a67 Fix harmless compiler warning in FTS5. 
FossilOrigin-Name: 02448a9fccd4532daa667e4ba6ff1dbb7f68008a
 2015-08-18 16:32:45 +00:00
dan 7899fa78c1 Minor optimization for fts5 API xInst(). 
FossilOrigin-Name: efb7c9c5d0015c8c966f8d6742c05cda82fc146a
 2015-08-12 15:36:58 +00:00
dan 7918dc835b Add the xPhraseFirst() and xPhraseNext() fts5 APIs, for faster iteration through a single phrases position list. Also optimize xInst() and xInstCount() a bit. 
FossilOrigin-Name: f7682435278419829a46bb4cc9b5625d46549e22
 2015-08-12 12:11:28 +00:00
mistachkin 7e968a7d0a Fix harmless compiler warnings in FTS5 code. 
FossilOrigin-Name: 0ddb2532b2daaaf1b0109ac360822f84cb999b7f
 2015-08-06 03:19:14 +00:00
mistachkin c38e00f61e Fix duplicate test numbering in the FTS5 matchinfo tests. 
FossilOrigin-Name: 483ebe8989ed7dbe197fe84e3eef6e76fa828e14
 2015-08-06 03:09:49 +00:00
dan 50b5491771 Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so. 
FossilOrigin-Name: 9e3aafe44a0813aa2a0c6172fdba1440b8a973ec
 2015-08-05 19:35:59 +00:00
dan f9419d17ff Remove all references to "docid" within fts5 source code and comments. Replace with "rowid". 
FossilOrigin-Name: dffd358f6cbf575d3b1045b1ce53429d15bade2a
 2015-08-05 07:43:46 +00:00
dan d3789c0028 Add new test file fts5_test_mi.c, containing an implementation of a function similar to FTS4 matchinfo() for FTS5. 
FossilOrigin-Name: 4f9520a9dc9c667b7fda5b0822de2bf48184ac99
 2015-08-04 20:29:00 +00:00
dan 90c3179bb4 Remove a duplicate "typedef struct Fts5Global Fts5Global;" from fts5_main.c (duplicate is in fts5Int.h). 
FossilOrigin-Name: 54a771fe2c2b3c1cc2914a527a825934fe114c49
 2015-07-31 17:48:07 +00:00
dan 79e2347fdf Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer. 
FossilOrigin-Name: c3c672af97edf2ae5d793f6fa47364370aa4f4ec
 2015-07-31 14:43:02 +00:00
dan dbac3ece0f Return an error message (instead of segfaulting) if test function fts5_expr() is invoked with no arguments. 
FossilOrigin-Name: 8e818b8985c0196cd9671a6491796faaeebeb16e
 2015-07-30 11:26:10 +00:00
dan 0f43df31b5 Update comments in fts5.h describing the xColumnCount() API function. 
FossilOrigin-Name: a623eb55d1583bcfe360a8dabd29f2a583c61e64
 2015-07-29 14:58:47 +00:00
dan 291d090947 Merge the fts5-btree-index branch with this one. FTS5 changes only. 
FossilOrigin-Name: 9f1dcdc3e1644c21762dacf619fac70afe6318c5
 2015-07-27 12:19:11 +00:00
dan c3b4132f53 Bump the "version" field in the fts5 config table. 
FossilOrigin-Name: 92c9178933ef95e92c343c0a741a76bbdf071966
 2015-07-27 11:49:01 +00:00
dan 8d90087146 Minor tweak to the SQL schema used by fts5. 
FossilOrigin-Name: 427bf09ba41158041d774aac2b5ffbd062c6eabe
 2015-07-27 11:43:36 +00:00
dan d0dc8cb5a3 Remove a potentially undefined behaviour involving signed integers and bitshift operations from fts5_hash.c. 
FossilOrigin-Name: ad6286ab1f40e4716456a67b719f9cd733f988eb
 2015-07-27 11:01:19 +00:00
dan 44198f30b5 Merge latest trunk changes with this branch. 
FossilOrigin-Name: 5ec933c257884019484db7f533ce920012e47a1c
 2015-07-27 10:46:48 +00:00
dan 08406bf064 Fix warnings in fts5 code. 
FossilOrigin-Name: bd4f156d07d0db1f063d42891965920fc2229352
 2015-07-24 20:34:40 +00:00
dan 3baaacae03 Merge trunk changes, including fixes for compiler warnings in fts5 code, with this branch. 
FossilOrigin-Name: 7190d79ba452ceb1af77ce1375278b097816a8be
 2015-07-16 20:24:42 +00:00
dan 8694d6049f Fix compiler warnings in fts5 code. 
FossilOrigin-Name: e9bf275cd969eca6fb41384d3637528d6a19f819
 2015-07-16 20:17:57 +00:00
dan e3229c19cb Use a WITHOUT ROWID table to index fts5 btree leaves. This is faster to query and only slightly larger than storing btree nodes within an intkey table. 
FossilOrigin-Name: 862418e3506d4b7cca9c44d58c2eb9dc915d75c9
 2015-07-15 19:46:02 +00:00
mistachkin 532f179cab Fix some harmless compiler warnings. 
FossilOrigin-Name: 3de030c904d125ccf41fa1929646b8a002b5341b
 2015-07-14 17:18:05 +00:00
dan 3eb9024d68 Fix inconsistencies in formatting of fts5 docs. 
FossilOrigin-Name: 5fb4c77163be8d5ff1e2934dda3cc2ad0a6267e8
 2015-07-10 17:55:42 +00:00
dan d2454f47e7 Improve the performance of docid merges in fts5. 
FossilOrigin-Name: b2de77a01cc5edcea2f98f7916e64cb33e6bd414
 2015-07-09 20:46:35 +00:00
dan 3dbd2397f4 Reduce the number of calls to malloc() made by fts5. 
FossilOrigin-Name: 898618ccf61d3ad166d9fc742e132d518338b5c3
 2015-07-09 19:02:19 +00:00
dan bcf2163771 Merge the Fts5MultiSegIter and Fts5IndexIter objects. 
FossilOrigin-Name: 6ace73541ae48aeb19e0c2d9f6772d032d0a7eae
 2015-07-07 19:35:11 +00:00
dan b2fe306a2e Further optimizations for fts5 b-tree seeks. 
FossilOrigin-Name: f37899686c032145f431f81c1072738536c61c7e
 2015-07-07 08:29:32 +00:00
dan 656773b675 Speed up seek operations on fts5 b-tree structures. 
FossilOrigin-Name: 7b7da1eb435d321fc4283f6aa2161fa1e16f2cf3
 2015-07-06 20:27:19 +00:00
dan 30ae503167 Optimize seek operations on fts5 b-trees. 
FossilOrigin-Name: 8cf02090ce53ec150492d77d9e5e5f27665bd34f
 2015-07-04 18:44:07 +00:00
dan 70c8885417 Fix a problem in the fts5 integrity-check. 
FossilOrigin-Name: 4e473559c9bdcf4d0b823e995a3642cd50ebd337
 2015-07-04 15:35:02 +00:00
dan 28a10c3c1f Rework the Fts5MultiSegIter structure a bit to make it more efficient. 
FossilOrigin-Name: 0778825d0ec9315c70659fae8d0640b209049dd8
 2015-07-03 20:47:18 +00:00
dan 00a1351e55 Speed up eof checks on fts5 cursors. 
FossilOrigin-Name: 3df4af5d8c28863783b0bc867abfbe31cc96f1b9
 2015-07-03 19:13:56 +00:00
dan ec7c1de354 Avoid freeing the array of instance matches each time the fts5 xNext() method is called. 
FossilOrigin-Name: b9b0c1e50d77f5d6e02f43fbb100c722cb692cc5
 2015-07-03 17:14:18 +00:00
dan a153bbc4c2 Copy fts5.h into the build directory as part of building fts5.c. Also remove code to edit fts5parse.c from various Makefiles and add it to mkfts5c.tcl. 
FossilOrigin-Name: 4d8f03dc7b6529cf8d46fc0fcc82112c0394dc13
 2015-07-02 20:12:34 +00:00
dan 6544b8643f Minor change to mkfts5c.tcl so that fts5.c can be used according to the instructions in loadext.html. 
FossilOrigin-Name: edbcccd349090efff3f975e27451608136a429fc
 2015-07-02 18:52:16 +00:00
dan 3abdc44e7b Add fts5_source_id() SQL function, similar to sqlite_source_id(). 
FossilOrigin-Name: ec2573487cb86664f7f056168a764c28ca8236fc
 2015-07-02 18:16:02 +00:00
dan 3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c. 
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
 2015-07-02 15:52:21 +00:00
dan 6394d99a0e Fix a segfault that could follow an OOM error in fts5. 
FossilOrigin-Name: 713239b8cf2900e8f7d97646c7f350248b4e804f
 2015-06-26 20:08:25 +00:00
dan 6fbeb22596 Fix some cases in the fts5 code where a corrupt database could cause a buffer overread. 
FossilOrigin-Name: 360c57bbb8bad6cc84af140f61e0282fa06462b4
 2015-06-26 18:50:29 +00:00
mistachkin a468027c8d Merge upstream changes. Get the test fixture working with FTS5. Fix compiler warnings. 
FossilOrigin-Name: bfcd16089cf8c65130c011a9718bf7812685c841
 2015-06-26 16:55:55 +00:00
dan eb37567fe2 Remove fts5 initialization code from core file main.c. 
FossilOrigin-Name: c91a93b343b4d3b1a19afdb5d23d1c508f9779d2
 2015-06-26 16:42:17 +00:00
mistachkin ed52f9ff48 Initial changes to get FTS5 working with MSVC. 
FossilOrigin-Name: ef2052f81e33ca98e85a60f8a78cdd19a7c1c35c
 2015-06-26 04:34:36 +00:00
dan 3e65f89ef6 Fix Makefile.in to account for recent changes. Update fts5 test scripts so that they do not run if SQLITE_ENABLE_FTS5 is not defined. 
FossilOrigin-Name: 3175220747f5d4dadd17e09fb1899dc782c90926
 2015-06-25 20:36:36 +00:00
dan ed2589f555 Add a script to combine all fts5 code into a single file - fts5.c - that can be used to build an SQLite loadable extension. 
FossilOrigin-Name: 46e86b0637248fb4d623c97778cc041eabe3636c
 2015-06-25 20:10:24 +00:00
dan 51ef0f57c7 Improve test coverage of fts5. 
FossilOrigin-Name: df5ccea80e8f0da83af5e595b539687006085120
 2015-06-23 18:47:55 +00:00
dan c94a08100b Merge latest trunk changes with this branch. Add tests for columnsize=0. 
FossilOrigin-Name: ef44c71a22518727030dd90c0139af8973b05841
 2015-06-23 15:06:13 +00:00
dan 1153e60cf0 Fix the fts5 xRename() method. 
FossilOrigin-Name: 0f7fd51325875fbf0f1eaca3bbbd170ef99c4208
 2015-06-10 10:45:34 +00:00
dan bcc2f04c68 Add the "columnsize=" option to fts5, similar to fts4's "matchinfo=fts3". 
FossilOrigin-Name: aa12f9d9b79c2f523fd6b00e47bcb66dba09ce0c
 2015-06-09 20:58:39 +00:00
dan 90502c3dc7 Fix a comment in fts5.h. 
FossilOrigin-Name: e964b5877497b16cf985d3d847e82529bb3fa4a3
 2015-06-06 19:23:32 +00:00
dan 7193b69f52 Fix handling of fts5 rowid constraints in the absence of a MATCH clause. Add tests to cover recently added branches. 
FossilOrigin-Name: 3a9cb648138a059862fb438c0787fdd888f5e88e
 2015-06-06 16:28:29 +00:00
dan e4449454c5 Make use of range constraints on the rowid field of an fts5 table in full-text queries. 
FossilOrigin-Name: 32cbc0ed3699cc21302f0b6a159493117ad4bd4f
 2015-06-05 19:05:57 +00:00
dan 6f277eb6bc Fix an fts5 problem in extracting columns from position lists containing large varints. 
FossilOrigin-Name: 4ea015ab983300d420ef104cca550b22a6395866
 2015-06-03 11:23:30 +00:00
dan f67bb4a21b Change the fts5 multi-column syntax to use parenthesis instead of square brackets. 
FossilOrigin-Name: ab85a6fc4f7580278fc9d1f0090fdcf0a90d065b
 2015-06-02 19:38:15 +00:00
dan 167fd757f0 Merge latest trunk changes with this branch. 
FossilOrigin-Name: c9ffda4abb4390bbc5719e269196e2807b254f97
 2015-06-02 18:07:58 +00:00
dan dbef6e752e Reimplement [ec69e09a] so that each call to the xNext() method does not involve two iterations of the match expression tree (only one). 
FossilOrigin-Name: 80fe305b3eefb17310a9d6185d1c8cd73ee38b1e
 2015-06-02 17:57:01 +00:00
dan 5350d91ed3 Improve performance of the fts5 AND operator. 
FossilOrigin-Name: b43e9a5b7a0483ccb102316a4dbc5e32b5bc69ec
 2015-06-01 19:17:06 +00:00
dan 34efc82eed Change fts5 expression processing to avoid linear scans of long doclists caused by phrases that match specific columns only. 
FossilOrigin-Name: ec69e09a55b4daf1c40aeaaf9ee95091fe86f5c0
 2015-06-01 09:15:20 +00:00
dan 3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h. 
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
 2015-05-30 11:49:58 +00:00
dan a8c024905c Add extra tests and fixes for multi-column matches. 
FossilOrigin-Name: ae6794ffa23ef6191bd8834422abf322d978c11b
 2015-05-29 19:00:22 +00:00
dan 6d21f42db2 Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>". 
FossilOrigin-Name: 0fc0ea20920615f3e48ea2dbe2b7dcd979b0993e
 2015-05-29 15:55:30 +00:00
dan e96efb6426 Optimizations for fts5 queries that match against a specific column. 
FossilOrigin-Name: b29ac50af0491a780a5a4c0985d88d0e5e014ba3
 2015-05-28 19:57:12 +00:00
dan af5cd564f0 Remove some dead code from fts5. Add auxiliary function api tests to the same. 
FossilOrigin-Name: 0f9df202cc58097afddb8dad662b7c7fdc2c7d0c
 2015-05-28 14:37:26 +00:00
dan 7f64db1343 Simplifications and minor optimizations to fts5 prefix queries that cannot use a prefix index. 
FossilOrigin-Name: aef89d9f6aa536efee347367558cf5d4ea81b238
 2015-05-26 18:22:01 +00:00
dan 494016a424 Avoid redundant loads from the %_data table in the fts5 code. 
FossilOrigin-Name: 02069782f8b7896a582582c79185b50418622736
 2015-05-25 11:46:33 +00:00
dan 2e7d35e2fe Avoid making redundant copies of position-lists within the fts5 code. 
FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0
 2015-05-23 15:43:05 +00:00
dan 0d45967b7f Increase test coverage of fts5_vocab.c. 
FossilOrigin-Name: 065ab83a6ce36e16d3b95a61505aa3cff0bfea84
 2015-05-22 07:44:44 +00:00
dan 21b7d2a9b8 Improve test coverage of fts5_unicode2.c. 
FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e
 2015-05-22 06:08:25 +00:00
dan 8c1f46de50 Improve test coverage of fts5_tokenize.c. 
FossilOrigin-Name: 0e91a6a520f040b8902da6a1a4d9107dc66c0ea3
 2015-05-20 09:27:51 +00:00
dan 116eaee4a0 Add tests for fts5 tokenizers. 
FossilOrigin-Name: 4f90ba20e2be6ec5755fe894938ac97342d6fbf6
 2015-05-19 19:37:09 +00:00
dan b10210ea1b Fix a memory leak that could follow an OOM condition in fts5. 
FossilOrigin-Name: de9f8ef6ebf036df5a558cd78fb4927da2d83ce8
 2015-05-19 11:32:01 +00:00
dan 09963b27b8 Improve test coverage of fts5_storage.c. 
FossilOrigin-Name: 4dfe2e5871704125338620705e6638f36baaacd7
 2015-05-18 20:34:59 +00:00
dan 23390b1b2b Fix a bug in test script fts5fault4.test. 
FossilOrigin-Name: a508059305641c2cb53197b0f6e93d7b723697e2
 2015-05-18 18:24:41 +00:00
dan 5211e54f95 Add a test for an untested branch in fts5_expr.c. 
FossilOrigin-Name: ce08206b5c5bf6a81decf20c99a524dd92c3715b
 2015-05-18 18:03:06 +00:00
dan 27aac274b9 Improve test coverage of fts5_config.c. 
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
 2015-05-18 17:50:17 +00:00
dan e0569a4aae Further test coverage improvements for fts5. 
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
 2015-05-16 20:04:43 +00:00
dan 369e4129b9 Improve test coverage of fts5_index.c. 
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
 2015-05-15 18:13:14 +00:00
dan 415ae725cb Add test cases. And some fixes. 
FossilOrigin-Name: adee788586197445672013d434e7ba47ce510b59
 2015-05-15 12:18:39 +00:00
dan 23d538885e Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token. 
FossilOrigin-Name: aa34bf666c384cf32a8d8166ab6d9afbca26a256
 2015-05-13 17:15:32 +00:00
dan 71ab324066 Allow the fts5vocab table to optionally provide data on a per-column basis. 
FossilOrigin-Name: 3922276135a7825d0ede8d9c757e9cfe492f803a
 2015-05-09 18:28:27 +00:00
dan 5e38f1c9bf Add the fts5vocab module, for direct access to the fts5 index. 
FossilOrigin-Name: 6bf93e3b56e6705b7d12bab5024fc615f373b36c
 2015-05-08 20:21:24 +00:00
dan 76724372ae Improve the error message returned by FTS5 if it encounters an unknown file format. 
FossilOrigin-Name: f369caec145f311bb136cf7af144e2695badcb9b
 2015-05-08 09:21:05 +00:00
dan 4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes. 
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
 2015-05-07 19:29:46 +00:00
dan 7c479d51e5 Reorganize some of the fts5 expression parsing code. Improve test coverage of the same. 
FossilOrigin-Name: c4456dc5f5f8f45f04e3bbae53b6bcc209fc27d5
 2015-05-02 20:35:24 +00:00
dan 0501b2492a Further improvements to test coverage of fts5 code. 
FossilOrigin-Name: d4331943dff259380c4025bb740d8aba6972d351
 2015-05-01 20:38:57 +00:00
dan 557085a5a1 Improve test coverage of fts5.c. 
FossilOrigin-Name: add4f4681c648dcbecaa68d08f7b2f4e6d63003c
 2015-05-01 12:14:23 +00:00
dan 7b2ec1ae41 Improve fts5 tests. 
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
 2015-04-29 20:54:08 +00:00
dan 90dd70226f Fix an fts5 bug in handling writes while there are active cursors. 
FossilOrigin-Name: 07f70955392697556ca2951c9b6c3a5204cd5ec0
 2015-04-28 20:24:50 +00:00
dan cdbd150170 Improve coverage of fts5 tests. 
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
 2015-04-28 18:35:28 +00:00
dan a3bdec7ee4 Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement. 
FossilOrigin-Name: e38e2bb637844dae8ae5d5f3e23d8369e1b91e45
 2015-04-27 16:21:49 +00:00
dan c1cea8f731 Further tests for fts5. 
FossilOrigin-Name: ffeb3ef3cfec3681b72bb28cfa612aa15e07887d
 2015-04-27 11:31:56 +00:00
dan 26c84373c4 Improve coverage of fts5_index.c slightly. 
FossilOrigin-Name: e5aaa01306597ffd2475dcb83ae889393f68d315
 2015-04-25 20:29:46 +00:00
dan b33e64c83e Add tests for fts5. 
FossilOrigin-Name: e748651c940eae2389fe826cf5c25f1166a5e611
 2015-04-25 18:56:48 +00:00
dan ef8b74324d Merge latest trunk changes with this branch. 
FossilOrigin-Name: 1c78d8920fb59da3cb97dd2eb09b3e08dfd14259
 2015-04-24 20:18:21 +00:00
dan df5bd1fed2 Add the "unindexed" column option to fts5. 
FossilOrigin-Name: 86309961344f4076ddcf55d730d3600ec3b6e45c
 2015-04-24 19:41:43 +00:00
dan def90aae18 Add extra tests for corrupt database handling in fts5. 
FossilOrigin-Name: 41449f7a0b5da6332eef48386c91ef63382c4783
 2015-04-24 15:56:09 +00:00
dan 206480992d Add extra OOM tests for fts5. 
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
 2015-04-22 20:58:24 +00:00
dan c51f5b3fd9 Fix problems with doclist-indexes involving very large rowids. 
FossilOrigin-Name: 1e5994097e4c740c5173ea9718c3935728fdb86f
 2015-04-22 20:14:46 +00:00
dan 5bb97c2b80 Fix a problem in fts5fault1.test. 
FossilOrigin-Name: a21d60cb2ac6463c012d82d1970d90da5da2a14a
 2015-04-21 20:13:38 +00:00
dan d7889b26ac Fix an fts5 problem with large deletes. 
FossilOrigin-Name: e50e8031d6f804ebe50e0eec9a8b6e7f0152ecc3
 2015-04-21 19:07:39 +00:00
dan 80d3ea080a Fix some fts5 problems with very large position lists. 
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
 2015-04-20 18:48:57 +00:00
dan 687c5124e0 Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances. 
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
 2015-04-15 18:49:20 +00:00
dan dbca0ab9b0 Fix a problem preventing doclist indexes from being loaded. 
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
 2015-04-15 16:01:42 +00:00
dan f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly. 
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
 2015-04-14 20:15:41 +00:00
dan 929b2cfc2e Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index. 
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
 2015-04-11 18:25:04 +00:00
dan 56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag. 
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
 2015-04-11 16:23:31 +00:00
dan 8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan. 
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
 2015-03-21 15:37:19 +00:00
dan f5fab92d82 Add an optimization to the fts5 unicode tokenizer code. 
FossilOrigin-Name: f5db489250029678fce845dfb2b1109fde46bea5
 2015-03-11 14:51:39 +00:00
dan c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees. 
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
 2015-03-10 19:24:30 +00:00
dan 92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions. 
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
 2015-03-07 15:46:41 +00:00
dan 6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want. 
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
 2015-03-07 11:50:31 +00:00
dan 47c467c80e Fix a couple of build problems. 
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
 2015-03-04 08:29:24 +00:00
dan f400282bdf Further minor optimizations to flushing fts5 data to disk. 
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
 2015-02-27 09:41:10 +00:00
dan 5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition. 
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
 2015-02-27 07:23:26 +00:00