Commit Graph

39 Commits

Author SHA1 Message Date
dan
24730de8d1 Add the fts5 'delete-automerge' integer option. A level is eligible for auto-merging if it has a greater than or equal percentage of its entries deleted by tombstones than the 'delete-automerge' option. Default value is 10.
FossilOrigin-Name: b314be66b9ac0190b5373b3b6baec012382bc588c2d86c2edab796669a4303c3
2023-07-24 19:13:06 +00:00
dan
2159292ce0 Integrate contentless delete with auto-merge.
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
2023-07-22 19:47:46 +00:00
dan
f7ff755651 Fix an assert() in fts5 that could fail with a corrupt database.
FossilOrigin-Name: e99979855de937ed5ee0994b180054501400bf8776fb70acd31786d2ba1ad49a
2021-10-16 13:59:08 +00:00
dan
f6b0464e36 Fix some fts5 problems with signed integer overflow causing segfaults in -ftrapv builds.
FossilOrigin-Name: 7e17c2f4b7dc9b563d0b4da949bb134dc7c4fc9c86ce03891432a884ca6409d5
2020-10-12 18:09:16 +00:00
dan
f2e151aeab Fix a problem in fts5 causing the in-memory hash table to be flushed more often than necessary.
FossilOrigin-Name: 9a55601d072bcd1ac0f6f1633216c7c8b3ff3e037451e2cfd285cbe19b60b9fc
2020-09-17 15:16:50 +00:00
mistachkin
065f3bf4f2 Fix various harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c
2019-03-20 05:45:03 +00:00
dan
d406d07262 Fix a buffer overread that could occur when running fts5 prefix queries inside a transaction.
FossilOrigin-Name: b3fa58dd7403dbd4d2e9f3ae23d7d1337830d6fef2aa2f137ac5174de0d5828e
2019-03-18 15:49:07 +00:00
dan
b15f19c75e Fix an fts5 problem with interleaving reads and writes in a single transaction.
FossilOrigin-Name: 45c73deb440496e848cb24d4c1326d4105dacfee8bbafb115e567051855e6518
2019-03-18 15:23:20 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
2019-01-08 20:02:48 +00:00
dan
d9d7d253fc Fix another potential buffer overread in fts5.
FossilOrigin-Name: d0e943d9574c7cab1af4a0fe753a0b5f91ea0e884b91a85b13239144b71f067c
2019-01-07 16:52:00 +00:00
mistachkin
831265f962 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: 57a60e959c198b87ed29fab14356f89e0ea7cdb1dd99adbea45b40e8d0cb310a
2017-05-24 15:32:44 +00:00
dan
22025ef198 Fix some problems in fts5 code detected by -fsanitize=undefined.
FossilOrigin-Name: 35f721045dfe3f82e016938ab1a668bfc37b6b57b8813cc963ef640ec82de58d
2017-05-19 12:32:32 +00:00
mistachkin
77fac879d2 More harmless compiler warning fixes.
FossilOrigin-Name: ab69527c1608da0b668f3b49e967661dd99cc3d4
2016-04-12 20:05:06 +00:00
dan
f2d328fa25 Fix a fairly obscure buffer overread in fts5.
FossilOrigin-Name: 130580207ab5cee762b2893808acef7c8afad027
2016-02-12 17:56:27 +00:00
dan
d11e397b2c Updates to fts5 to support detail=none mode. As of this commit, many cases are still broken.
FossilOrigin-Name: ac8f4cf0cede6bcbe47eeefb85d80a27e9278212
2015-12-30 19:58:57 +00:00
dan
9f44deed93 Change the name of the offsets=0 option to "detail=column". Have the xInst, xPhraseFirst and other API functions work by parsing the original text for detail=column tables.
FossilOrigin-Name: 228b4d10e38f7d70e1b008c3c9b4a1ae3e32e30d
2015-12-28 19:55:00 +00:00
dan
b12dc84fbb Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch.
FossilOrigin-Name: 40b5bbf02a824ca73b33aa4ae1c7d5f65b7cda10
2015-12-17 20:36:13 +00:00
mistachkin
b9becaa268 Fix even more harmless compiler warnings.
FossilOrigin-Name: 1d0e6aa119da8e15d35508f5d75ffc729979da92
2015-12-16 23:30:30 +00:00
mistachkin
cdabd7bd50 Fix harmless compiler warnings.
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
2015-10-14 20:34:57 +00:00
dan
d0dc8cb5a3 Remove a potentially undefined behaviour involving signed integers and bitshift operations from fts5_hash.c.
FossilOrigin-Name: ad6286ab1f40e4716456a67b719f9cd733f988eb
2015-07-27 11:01:19 +00:00
dan
8694d6049f Fix compiler warnings in fts5 code.
FossilOrigin-Name: e9bf275cd969eca6fb41384d3637528d6a19f819
2015-07-16 20:17:57 +00:00
dan
3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
2015-07-02 15:52:21 +00:00
dan
3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h.
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
2015-05-30 11:49:58 +00:00
dan
4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
2015-05-07 19:29:46 +00:00
dan
cdbd150170 Improve coverage of fts5 tests.
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
2015-04-28 18:35:28 +00:00
dan
f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly.
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
2015-04-14 20:15:41 +00:00
dan
56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
dan
92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions.
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
dan
6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
dan
47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan
5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
dan
dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments.
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
dan
6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
dan
5082140311 Minor optimizations to fts5 writes.
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
dan
719b0f66e5 Fix some problems with transactions that both read and write an fts5 table.
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
dan
8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan
0b520cc5f9 Add further tests and fixes for fts5.
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
2015-01-22 19:13:08 +00:00
dan
e2fb318e34 Automatically resize the hash table used by fts5.
FossilOrigin-Name: f1cb48f412a5f200f1fe04f91072864f379db08f
2014-08-12 08:36:00 +00:00
dan
c7fe7a969b Replace the hash table borrowed from fts3.
FossilOrigin-Name: 617e2fac1c128212254f71b1a8fddaf0d1d90262
2014-08-11 19:44:52 +00:00