mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,205 Commits 1,487 Branches 166 Tags 416 MiB
83a635a267
Commit Graph

254 Commits

Author SHA1 Message Date
dan
03155f659f Update fts5 to support "<colset> : ( <expr> )" for column filtering, as well 
as "<colset> : NEAR(...)" and "<colset> : <phrase>".

FossilOrigin-Name: c847543f8bb1376fef52bca72b4191162a32eb7e6c5f0cd1aa0ab116b3183396
 2017-04-12 17:50:12 +00:00
dan
be0bc8bb4d Have fts5 close any open blob-handle when a new savepoint is opened. This 
ensures that fts5 does not prevent DROP TABLE statements (which always open a
savepoint) from succeeding.

FossilOrigin-Name: a921ada89050ce1d162fd1b0056939573635e2cec7ac0c2a99ae924b3ae593f7
 2017-04-08 09:12:20 +00:00
dan
1439a1e80f Improve handling of corrupt data in fts5. 
FossilOrigin-Name: 609ac1c73f7c2b48a571c178a72383996a9538fe
 2017-01-06 11:55:03 +00:00
dan
b3e845ba33 Fix an fts5 problem causing a crash in phrase queries where the first token of 
the phrase matches one or more rows but some other token within the phrase
matches zero.

FossilOrigin-Name: e78f6f3bbf4781850960fe6741f7cba480f2c27d
 2016-11-14 08:19:37 +00:00
dan
460d38f15e Fix an fts5 problem with corrupt database handling found by address-sanitizer. 
FossilOrigin-Name: e22252e1da4cd9e41b970970a1c4f466aa6cc133
 2016-08-13 10:34:12 +00:00
drh
c73bd0a5cc Fix a harmless compiler warning in FTS5. 
FossilOrigin-Name: 2ea0c8b46eefd4874f021f8dfd344be751f2034f
 2016-08-11 01:24:56 +00:00
dan
882ef0b8c0 Have fts5 interpret column lists that begin with a "-" character as "match any column except" lists. 
FossilOrigin-Name: e517545650631d1e8a7ee63c6646a8b183a0a894
 2016-08-09 19:26:57 +00:00
mistachkin
77fac879d2 More harmless compiler warning fixes. 
FossilOrigin-Name: ab69527c1608da0b668f3b49e967661dd99cc3d4
 2016-04-12 20:05:06 +00:00
drh
997de998a8 Remove an unused local variable from FTS5. 
FossilOrigin-Name: 0ed693c29f184223cde3b3d51f0e06273e586803
 2016-03-23 15:53:45 +00:00
dan
848b190e40 Explicitly limit the size of fts5 tokens to 32768 bytes. 
FossilOrigin-Name: 70fc69eed9b09159899d7cbd1416a59d04210a63
 2016-03-23 15:04:00 +00:00
dan
0a2f2b546a Fix some errors in fts5 test scripts. 
FossilOrigin-Name: e1ab2d376a72786098125a41c1ea8140fcbd15c6
 2016-03-22 15:01:54 +00:00
drh
13969f5af0 Fix harmless compiler warnings. 
FossilOrigin-Name: 25d776e4523aefeec007943fe29aa17c23ccb301
 2016-03-21 22:28:51 +00:00
dan
f5268268b7 Have fts5 cache the structure of its index in main memory. Use "PRAGMA data_version" to figure out when this cache should be invalidated. 
FossilOrigin-Name: 902f221754f3cc6fb4ae049c135f85efce604ed3
 2016-03-21 15:30:50 +00:00
dan
3c6707f72b Rearrange code so that tests pass whether SQLITE_DEBUG is defined or not. 
FossilOrigin-Name: 89296a46c3b891ee0b637ff6f89d828d6e46e7b5
 2016-03-21 15:18:13 +00:00
dan
ef480d37dc Change the way fts5 internally allocates segment ids in order to eliminated non-determinism from the module. 
FossilOrigin-Name: d6e2637df16764aa9723a30ea2eb8a631d28cb2b
 2016-03-21 09:56:19 +00:00
dan
c9460ab989 Fix some OOM-handling issues in the fts5 changes on this branch. 
FossilOrigin-Name: 020a0bda59fe93b9361ceeed0d5a8ba4177380c8
 2016-03-17 12:39:56 +00:00
dan
f55fb6615b Have fts5 cache the decoded structure of fts5 indexes in memory. Use "PRAGMA data_version" to detect stale caches. 
FossilOrigin-Name: 33ef2210ef19e55c8d460bfe9d3dc146034c8acc
 2016-03-16 19:48:10 +00:00
dan
1bb85f3ed3 Add tests to cover untested branches in recent fts5 changes. 
FossilOrigin-Name: fa0033edf6ddf3c6631fa95b343225dbc8ac9429
 2016-03-10 15:12:47 +00:00
drh
6459ca0b8c Remove an unused local variable. 
FossilOrigin-Name: 3c343c3d012367942037e64a9855b825cb459844
 2016-03-10 14:22:42 +00:00
dan
4dbc65b29a Add an incremental optimize capability to fts5. Make the 'merge' command independent of the 'automerge' settings. 
FossilOrigin-Name: 556671444c03e3afca072d0f5e9bea2657de6fd3
 2016-03-09 20:54:14 +00:00
dan
4e9d0d5418 Fix a problem in fts5 where a corrupt db could lead to a (huge) buffer overread. 
FossilOrigin-Name: c9a30e117f2c6c9ef0cc0c6ca5227d2961715b8f
 2016-02-29 17:34:16 +00:00
dan
437a54eaba Fix an fts5 problem causing 'optimize' to corrupt the fts index under some circumstances. 
FossilOrigin-Name: 251d6473f7c9ad95adbdcc49cb8eaf7c0956764b
 2016-02-29 17:16:26 +00:00
drh
7ff89807e0 Fix a harmless compiler warning in FTS5. 
FossilOrigin-Name: e52c90758c53cbb0f74e0d8582e4c3c1bee04150
 2016-02-15 23:46:36 +00:00
drh
5d2e2f1267 Remove the unused fts5BlobCompare() routine. 
FossilOrigin-Name: defc762dd11144402abd591278819768a080d5ea
 2016-02-11 21:53:57 +00:00
dan
2a44b3d877 Avoid a buffer overread when reading a corrupt fts5 structure record. 
FossilOrigin-Name: facbc424e555061135aced7b134bf6c19f54e484
 2016-02-11 18:08:38 +00:00
dan
22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code. 
FossilOrigin-Name: bc3f7900d5a06829d123814a5ac7b951bcfc1560
 2016-02-11 17:01:32 +00:00
dan
d120a1e3ab Add tests for and remove unreachable branches from fts5 in order to restore test coverage. 
FossilOrigin-Name: 22589018ac3321f7bd89ce9fa69e57eae395e324
 2016-02-05 21:09:26 +00:00
dan
b80b8f482c Further streamlining of fts5 prefix query code. 
FossilOrigin-Name: ca91bd8ac70a5b3fef127364f73ec675e58bb92c
 2016-02-05 19:18:02 +00:00
dan
4786cf5a20 Further improve performance of unindexed fts5 prefix queries. 
FossilOrigin-Name: c9c6457d8ea911f6cc63967127e58da3146fd3ef
 2016-02-04 19:45:19 +00:00
dan
eb715c62f9 Improve performance of fts5 prefix queries on detail=col tables. 
FossilOrigin-Name: ca11f46db047e7f131cef3893f73824758a2076b
 2016-02-03 20:04:59 +00:00
dan
735ff4a809 Enhance the performance of fts5 AND and OR queries. 
FossilOrigin-Name: 62ea9e5ab8bc1a20245beebceb5ea62dcd7ec84e
 2016-02-02 17:40:41 +00:00
dan
52f5d1ba6b Performance enhancement for fts5 column filter queries on detail=full tables. 
FossilOrigin-Name: 13fb4aa6a87c5c6258979953da82eedc1a7bf14f
 2016-01-30 19:16:11 +00:00
dan
18205c781d Further minor performance improvements and code-size reductions related to fts5 column filters on detail=col tables. 
FossilOrigin-Name: b4ac61aeee976296e7719949cd4fb496147a29e8
 2016-01-26 20:08:50 +00:00
dan
1c16bccb79 Improve the performance of fts5 column filters on detail=col tables. 
FossilOrigin-Name: 249a2d070c34bf884a04cb248b9691e239f2871c
 2016-01-26 19:30:49 +00:00
dan
e9eb1593f5 Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode. 
FossilOrigin-Name: 72d53699bf0dcdb9d2a22e229989d7435f061399
 2016-01-23 18:51:59 +00:00
dan
8498f387be Fix a problem involving detail=col and column filters. 
FossilOrigin-Name: 7558a0ad2276e91f2faced8ea405d9fdb4fa0c6e
 2016-01-23 14:45:36 +00:00
dan
795502d26c Experimental performance enhancements for fts5. 
FossilOrigin-Name: b5a57b812fd6a734cf2a342bf0b730ae18912d73
 2016-01-22 19:48:34 +00:00
dan
6d3c2889ec Add tests for fts5. Fix a crash that can occur in fts5 if the database content is corrupted. 
FossilOrigin-Name: acaf426449bf6fd3140fd63141750ff69d1119a5
 2016-01-18 17:48:28 +00:00
dan
0536a07c0e Add a debugging function to print human-readable versions of fts5 detail=none leaf pages. 
FossilOrigin-Name: 8358af3658d888516cdef5f8c8d89e9bdee53f91
 2016-01-18 09:08:56 +00:00
mistachkin
1e38927c12 Fix a couple C99-isms in FTS5 that cause compile errors on MSVC. 
FossilOrigin-Name: 584848d4396d7755027fab377c97e40fbcc90d21
 2016-01-15 05:39:55 +00:00
drh
0ea94db612 Yet another change to FTS5 trying to get it to merge successfully into sessions. 
FossilOrigin-Name: 8dedff3b9ac3e6bf9c131fee19f7d26dc1ebd61f
 2016-01-14 14:33:36 +00:00
dan
17535393b9 Fix another compiler warning in fts5 code. 
FossilOrigin-Name: d17bc2c92f4d086280e49a3cc72993be7fee2da7
 2016-01-14 14:19:50 +00:00
dan
f705e9deab Fix compiler warnings in fts5. 
FossilOrigin-Name: 5a343cc0336bba056df4449e6cd2e3fb9e75a105
 2016-01-14 14:15:54 +00:00
dan
fd7601a125 Improve performance of fts5 low level iterators. 
FossilOrigin-Name: bc5118f40a11f64ffb4e1c086277fa80b9764745
 2016-01-12 19:28:51 +00:00
dan
4364e5c9f7 Fix an fts5 problem involving detail=none, "ORDER BY rowid DESC" and deleted items. Also add tests to verify that the documented operator precedences are correct. 
FossilOrigin-Name: 8d05cfd48db1462dfd3efd502d53bd33214403fb
 2016-01-06 18:53:57 +00:00
dan
fe0c3cfee1 Changes to run many fts5 tests with detail=none and detail=col tables as well as the default detail=full. Also fixes for the bugs uncovered by running said tests. 
FossilOrigin-Name: 6322a1d984e7946735bace8a069ef24b31754b3b
 2016-01-02 19:01:56 +00:00
dan
811501e237 Fix the fts5 integrity-check code so that it works with detail=none tables. 
FossilOrigin-Name: 3a1df69e58e5830da1dff158eedbe3817743f58f
 2015-12-31 18:39:14 +00:00
dan
841243156f Fix some problems with fts5 detail=none tables. Some still remain. 
FossilOrigin-Name: 6a6f7bc40d6b4c8a4a254a9098c9d2f31fde69ca
 2015-12-31 17:36:58 +00:00
dan
d11e397b2c Updates to fts5 to support detail=none mode. As of this commit, many cases are still broken. 
FossilOrigin-Name: ac8f4cf0cede6bcbe47eeefb85d80a27e9278212
 2015-12-30 19:58:57 +00:00
dan
9a3a5275cd Add the xPhraseFirstColumn() and xPhraseNextColumn() API functions to fts5. For iterating through the set of columns that contain intances of a phrase. 
FossilOrigin-Name: 8c30605bcd0a78a5015948171145bc6f640b8358
 2015-12-29 19:35:03 +00:00
dan
9f44deed93 Change the name of the offsets=0 option to "detail=column". Have the xInst, xPhraseFirst and other API functions work by parsing the original text for detail=column tables. 
FossilOrigin-Name: 228b4d10e38f7d70e1b008c3c9b4a1ae3e32e30d
 2015-12-28 19:55:00 +00:00
dan
159fd77e0f Fix an fts5 integrity-check problem that affects offsets=0 tables with prefix indexes. 
FossilOrigin-Name: 609a0bc7f34e6dae74ce756aff920f3df78fe828
 2015-12-21 18:45:09 +00:00
dan
c58b9eeaaa Fix a problem with prefix queries on fts5 offsets=0 tables. 
FossilOrigin-Name: ad0987d83c252dd8d6a69321893629d7be805c28
 2015-12-18 19:07:14 +00:00
dan
b12dc84fbb Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch. 
FossilOrigin-Name: 40b5bbf02a824ca73b33aa4ae1c7d5f65b7cda10
 2015-12-17 20:36:13 +00:00
mistachkin
b9becaa268 Fix even more harmless compiler warnings. 
FossilOrigin-Name: 1d0e6aa119da8e15d35508f5d75ffc729979da92
 2015-12-16 23:30:30 +00:00
dan
dbbda39453 Have fts5 load its configuration from the xConnect() method is invoked. This ensures that the very first query run uses the correct value of the 'rank' option. 
FossilOrigin-Name: 33e6606f5e497e81119ec491cf2370f60bddafc0
 2015-11-06 12:50:57 +00:00
dan
d82211db56 Add the 'hashsize' configuration option to fts5, for configuring the amount of memory allocated to the in-memory hash table while writing. 
FossilOrigin-Name: 445480095e6877cce8220b1c095f334bbb04c1c3
 2015-11-05 18:09:16 +00:00
dan
24b8457911 Fix problems with prefix queries in fts5. 
FossilOrigin-Name: 3b5758c647530bd5c2e68d0ee3e9f58a96347ca4
 2015-10-27 17:48:57 +00:00
mistachkin
16158eeb49 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: aa4e01ea1af327d1f8398ebea1c5bacc46698c3d
 2015-10-21 22:08:36 +00:00
dan
99b25309d0 Remove some branches made unreachable by recent changes from fts5. 
FossilOrigin-Name: ae350bfbfd59f912c469a51bf7d1409f3c8d958a
 2015-10-21 20:56:27 +00:00
dan
f2c9995dbe Minor optimization for fts5 queries. 
FossilOrigin-Name: 363b36d50b6f3e3d3e79c538be1167b071a5a710
 2015-10-21 20:07:08 +00:00
mistachkin
6d7734c27d Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 0a903ec26bfbbe9bfd18bc34934e689ffdb01ce5
 2015-10-20 23:27:14 +00:00
dan
219f4d48a2 More optimizations for fts5 prefix queries. 
FossilOrigin-Name: b8fb263ed1b36d085437131795505e53af103d26
 2015-10-20 21:05:53 +00:00
dan
9cc4ae6f2a Fix #line directives added to generated file fts5.c. 
FossilOrigin-Name: 1f5f5804cd394a81b1c0712ce6d1052f228d0a40
 2015-10-20 19:55:35 +00:00
dan
8789fbb2bd Another optimization for fts5 prefix (and other) queries. 
FossilOrigin-Name: 60a8bde055a960c5b8cb4e231802c75617c942d8
 2015-10-19 20:49:10 +00:00
dan
5e5b454cd8 Another tweak to improve performance of fts5 prefix queries. 
FossilOrigin-Name: 69be427c864dc3de804ca3c5f1c6addcd33cd188
 2015-10-19 17:43:24 +00:00
dan
05903e6a87 Handle equality and range constraints on the "term" column of fts5vocab tables. Also have the same module report columns using names instead of indexes. 
FossilOrigin-Name: 6020d96ab47a0bebe41f691a3f44fd55d8b2d2b7
 2015-10-17 19:49:01 +00:00
dan
5f93ccd7fc Tests and minor fixes to improve coverage of FTS5 code. 
FossilOrigin-Name: f4de6d450e143cb1dd5447800d4b178e3db114c8
 2015-10-14 21:08:48 +00:00
mistachkin
cdabd7bd50 Fix harmless compiler warnings. 
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
 2015-10-14 20:34:57 +00:00
dan
df705d8185 Further tweaks to improve fts5 prefix query performance. 
FossilOrigin-Name: 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
 2015-10-12 19:12:29 +00:00
dan
98e713bb8d Further optimizations for fts5 prefix queries without a prefix index. 
FossilOrigin-Name: 83dc1ff7fa010715ca7f406a572f4ee444a967d7
 2015-10-07 19:06:21 +00:00
dan
22c7cca21f Allow the subsitute character (codepoint 26 - 0x1A) to appear in fts5 barewords. 
FossilOrigin-Name: baec1b96cb64a6a6317143a8df841e502afe3914
 2015-10-07 17:01:22 +00:00
dan
d46cc33131 Add tests for fts5 phrase queries with column filters. 
FossilOrigin-Name: f20f9f813f00cefcd427e506a09b0b973c782e3f
 2015-10-07 13:24:27 +00:00
dan
92099b0753 Add further tests for fts5 prefix queries. 
FossilOrigin-Name: accdc98b1291f07b802fd23f3ebc7dbc02ba09d3
 2015-10-07 09:02:50 +00:00
dan
a2507137f3 Optimizations for fts5 expressions that filter on column. More still to come. 
FossilOrigin-Name: bf1607ac155018573ca40fb58aca62c5fea7e60b
 2015-10-06 20:53:26 +00:00
dan
6f832d3dff Improve performance of prefix queries without a prefix index on fts5 tables. 
FossilOrigin-Name: f2f0184e9e1c9f121ee2ac864cd28c8cd8efecb5
 2015-10-05 19:41:16 +00:00
dan
bca189d5d8 Update fts5 to avoid using a statement journal for UPDATE and DELETE operations that affect at most a single row. 
FossilOrigin-Name: 5c83b9db46d61cfa76a1abed50467e2f02db0eb0
 2015-10-02 20:04:30 +00:00
dan
4ad0971919 Fix an off-by-one error in test function fts5_decode(). 
FossilOrigin-Name: 3a9f076250d9559d8ea94ba44095ba3ddbc5542d
 2015-09-29 12:19:51 +00:00
dan
6609bab24d Add test cases to cover fts5 integrity-check code. 
FossilOrigin-Name: 1d018c35b9e81982df036f5e62a4a42219b54e02
 2015-09-15 14:39:17 +00:00
dan
128ea696ea Fix a problem with fts5 "ORDER BY rowid DESC" queries and large terms. 
FossilOrigin-Name: b26d8f79c62cb71b4c3ea5df9d0e09558bbbd794
 2015-09-15 11:58:33 +00:00
dan
dafeda754b Improve test coverage of fts5_index.c. 
FossilOrigin-Name: c1f76686cee3918b1be785a4071d68cb3afda0ef
 2015-09-14 19:51:05 +00:00
mistachkin
16a93128f8 Fix harmless compiler warnings. 
FossilOrigin-Name: bfc7b84b766860d2e410702ba7c1166d7328309a
 2015-09-11 18:05:01 +00:00
dan
bcb04b6128 Fix some compiler warnings in fts5 code. 
FossilOrigin-Name: 0dc436116e55e3fd55eb6085ada71e099069b32d
 2015-09-11 14:15:46 +00:00
dan
56c8634853 Fix a segfault in fts5 that could occur if the database contents were corrupt. 
FossilOrigin-Name: 4931e37da4d2c26d7afc5432f7f0d534b51a85fa
 2015-09-10 16:19:01 +00:00
dan
b97efa0ba3 Update description of on-disk format in fts5_index.c. 
FossilOrigin-Name: 85aac7b8b6731e2f6880b80cfd62d431ea059799
 2015-09-10 15:49:16 +00:00
dan
f679d97d18 Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages. 
FossilOrigin-Name: ffe2796ac9244c62325fce4960f26c653321623c
 2015-09-10 10:01:30 +00:00
dan
a5aa8e1db2 Change the array of 16-bit offsets at the end of each page to an array of varints. 
FossilOrigin-Name: fab245bea4f283714c17bca22428d5eb4db5935a
 2015-09-10 05:40:17 +00:00
dan
fe8e2eba0a Remove the 0x00 terminators from the end of doclists stored on disk. 
FossilOrigin-Name: 00d990061dec3661b0376bd167082942d5563bfe
 2015-09-08 19:55:26 +00:00
dan
60f8139a96 Use macros to make the code in fts5_index.c easier to read. 
FossilOrigin-Name: 67ff5ae81357eb7fa28049bb724a22cb6f52e076
 2015-09-07 08:14:30 +00:00
dan
e386a1ba25 Experiment with a different fts5 leaf page format that allows faster seeks. 
FossilOrigin-Name: a1f4c3b543eed84e808f6b901a38179786fffe16
 2015-09-05 19:52:08 +00:00
dan
4dfe98a8c4 Enhance showfts5.tcl so that it can optionally display the number of terms in each segment. 
FossilOrigin-Name: d648ddd93de039820f5abe064c7bc1318cd9d6b1
 2015-09-04 11:13:00 +00:00
dan
e1c28d7229 Remove some more code from fts5_index.c by consolidating similar functions. 
FossilOrigin-Name: 59ae30b97b40faa363c55aa2664dead9eaeeddc0
 2015-09-03 14:22:27 +00:00
dan
5d144875bc Remove dead code from fts5_index.c. 
FossilOrigin-Name: 8a0a9b01e74072ee52fe393311ad591208fbbf7c
 2015-09-03 11:17:52 +00:00
mistachkin
7e968a7d0a Fix harmless compiler warnings in FTS5 code. 
FossilOrigin-Name: 0ddb2532b2daaaf1b0109ac360822f84cb999b7f
 2015-08-06 03:19:14 +00:00
dan
50b5491771 Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so. 
FossilOrigin-Name: 9e3aafe44a0813aa2a0c6172fdba1440b8a973ec
 2015-08-05 19:35:59 +00:00
dan
f9419d17ff Remove all references to "docid" within fts5 source code and comments. Replace with "rowid". 
FossilOrigin-Name: dffd358f6cbf575d3b1045b1ce53429d15bade2a
 2015-08-05 07:43:46 +00:00
dan
291d090947 Merge the fts5-btree-index branch with this one. FTS5 changes only. 
FossilOrigin-Name: 9f1dcdc3e1644c21762dacf619fac70afe6318c5
 2015-07-27 12:19:11 +00:00
dan
8d90087146 Minor tweak to the SQL schema used by fts5. 
FossilOrigin-Name: 427bf09ba41158041d774aac2b5ffbd062c6eabe
 2015-07-27 11:43:36 +00:00
dan
44198f30b5 Merge latest trunk changes with this branch. 
FossilOrigin-Name: 5ec933c257884019484db7f533ce920012e47a1c
 2015-07-27 10:46:48 +00:00
dan
08406bf064 Fix warnings in fts5 code. 
FossilOrigin-Name: bd4f156d07d0db1f063d42891965920fc2229352
 2015-07-24 20:34:40 +00:00