Commit Graph

57 Commits

Author SHA1 Message Date
dan 56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
dan 8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
2015-03-21 15:37:19 +00:00
dan c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees.
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
2015-03-10 19:24:30 +00:00
dan 92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions.
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
dan 47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan f400282bdf Further minor optimizations to flushing fts5 data to disk.
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
2015-02-27 09:41:10 +00:00
dan 5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
dan dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments.
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
dan 4a7e11c629 Fix an fts5 bug in large incremental merges.
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
2015-02-26 14:54:03 +00:00
dan 6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
dan 57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation.
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
dan 5082140311 Minor optimizations to fts5 writes.
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
dan 719b0f66e5 Fix some problems with transactions that both read and write an fts5 table.
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
dan 900f1922ac Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
2015-01-27 20:41:00 +00:00
dan 8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan 641cb4360a Fix compression of keys stored on internal segment b-tree nodes by fts5.
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
2015-01-23 17:43:21 +00:00
dan 626d9e3062 Remove some redundant code from fts5.
FossilOrigin-Name: 939b7a5de25e064bdf08e03864c35ab718da6f6f
2015-01-23 06:50:33 +00:00
dan 0b520cc5f9 Add further tests and fixes for fts5.
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
2015-01-22 19:13:08 +00:00
dan 4c2871bead Further tests and fixes for fts5.
FossilOrigin-Name: c020a291ed293a66d21c5885e50a7fee04aa6366
2015-01-21 20:30:14 +00:00
dan dd8a501019 Fix an fts5 issue with loading doclist-indexes for a term that is the last thing on its leaf page.
FossilOrigin-Name: e0d614425f7f5ffe266fdc03642931b1ec19ad25
2015-01-21 18:23:25 +00:00
dan b461380333 Add further tests for fts5 backend.
FossilOrigin-Name: 09dabb3b9e140eec6cfda83bcb86b6b9f5cf54b6
2015-01-21 16:10:59 +00:00
dan fd0b436e10 Add extra fault injection tests to fts5.
FossilOrigin-Name: f45a0dc0a8911c8aac5a1028ac4f543a709656e7
2015-01-20 20:34:17 +00:00
dan d8736bc3b8 Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration.
FossilOrigin-Name: f30afd209aa4ce42766b1493750c4f5b5f1e9502
2015-01-17 20:01:52 +00:00
dan 851ca6e715 Fix prefix indexes so that they work in characters, not bytes.
FossilOrigin-Name: af8d43a4a08528bbae25ee38fe25de8a86f8a21c
2015-01-13 17:25:08 +00:00
dan d6b1c880d2 Add the fts5 'optimize' command.
FossilOrigin-Name: e749be563d8e738af113bd301770e2f22763ab77
2015-01-07 19:33:11 +00:00
dan 96a32bb51c Add the 'rebuild' and 'delete-all' commands.
FossilOrigin-Name: 0cb2fed525778d96237b5b0943047665e1f636d1
2015-01-07 17:11:11 +00:00
dan aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
2015-01-06 19:08:26 +00:00
dan 2a28e507f7 Further fixes and test cases related to external content tables.
FossilOrigin-Name: ce6a899baff7265a60c880098a9a57ea352b5415
2015-01-06 14:38:34 +00:00
dan 2ae0f4c931 Fix a problem with prefix queries and the AND operator.
FossilOrigin-Name: 38b3c65e3ee95eb7afadb76e0110570fbbc41e1b
2014-12-18 20:01:15 +00:00
dan e4bec37900 Fix various problems in fts5 revealed by fault-injection tests.
FossilOrigin-Name: e358c3de5c916f2c851ab9324ceaae4e4e7a0fbd
2014-12-18 18:25:48 +00:00
dan 845d0ab323 Begin testing fts5 OOM and IO error handling.
FossilOrigin-Name: 2037dba62fdd995ad15b642abe499a790f5ffe5c
2014-12-03 17:27:35 +00:00
dan ca7fad3d2c Add a cookie mechanism to ensure that the %_config table is re-read as required.
FossilOrigin-Name: bb4a37b53de60da9ec8b9317eec14afa99690828
2014-11-28 20:01:13 +00:00
dan cb11e73fc2 Add a %_config table to fts5.
FossilOrigin-Name: 83491c56661ca78f96020ba68184bb3fb19e674f
2014-11-27 20:03:45 +00:00
dan cb62aae034 Add the auxiliary highlight() function to fts5.
FossilOrigin-Name: 059092379f981eb919b500ce447006f9e645fc5a
2014-11-24 16:24:33 +00:00
dan 6885bbc713 Add an "automerge=0" mode that disables auto-merging and falls back to fts4-style crisis merges.
FossilOrigin-Name: 2397404e152b908d838e6491294b263b05943b3f
2014-08-18 19:30:01 +00:00
dan e2fb318e34 Automatically resize the hash table used by fts5.
FossilOrigin-Name: f1cb48f412a5f200f1fe04f91072864f379db08f
2014-08-12 08:36:00 +00:00
dan 1a669f84a1 Simplify the way position lists are copied when merging data.
FossilOrigin-Name: 9f8d678a0ea75e169daf8b3f00bd05f52a050ea6
2014-08-11 20:26:34 +00:00
dan c7fe7a969b Replace the hash table borrowed from fts3.
FossilOrigin-Name: 617e2fac1c128212254f71b1a8fddaf0d1d90262
2014-08-11 19:44:52 +00:00
dan b191db8390 Use multiple memory allocations for a single Fts5Structure object. This is probably less efficient but much easier to get right.
FossilOrigin-Name: 2821825f7a481755a333dcdcad780b3e24448f20
2014-08-09 18:02:27 +00:00
dan 1d3ed1f46a Add "segment promotion" to fts5. This prevents the FTS index from growing indefinitely as data is added and deleted.
FossilOrigin-Name: ba359d78e166d78e0dc89e3c63a9a41e9ffea989
2014-08-07 18:47:33 +00:00
dan d90aab8f94 Avoid writing delete markers to the oldest segment in an FTS index.
FossilOrigin-Name: 1baeb1cee61d9c56c718b50af034a24f1018a322
2014-08-06 20:04:14 +00:00
dan 7b71fbaf93 Add support for savepoints to fts5.
FossilOrigin-Name: 3b19eba042bb2eeb1be60f8d58ebaa0a045d6a5c
2014-08-06 16:30:21 +00:00
dan d5997d80cb Use doclist-indexes with "ORDER BY rowid ASC" fts5 queries as well.
FossilOrigin-Name: d028ba6589f3122b635474c2683c0f93d5bc6c7c
2014-08-05 19:00:22 +00:00
dan 9a67058a63 Fix fts5_index.c to use doclist-indexes when possible. Only some cases work so far.
FossilOrigin-Name: 90b82d3ef613b2915e0e280dc1d2e5a2b617d59c
2014-08-04 20:07:40 +00:00
dan 9af0705e84 Start changing things to use doclist indexes as required. code is not activated yet.
FossilOrigin-Name: b8864da95db2c0e611116304d607e35a86c9247d
2014-08-02 20:49:36 +00:00
dan ff31da03e5 Add a special case to the integrity-check code to check that the final integer in a doclist index is as expected.
FossilOrigin-Name: c98934155cb48adfda57bd0fd1b950226d45f67a
2014-08-01 20:13:49 +00:00
dan d34742901d Have the fts5 integrity-check verify that doclist indexes match the contents of the leaf pages that they index.
FossilOrigin-Name: 37a7d3035eb4bbad7e32fe550321ac9fae611a57
2014-08-01 19:27:07 +00:00
dan a29284d65f Add "doclist index" records to the database. These are to make navigating within very large doclists faster. They are not yet used by queries.
FossilOrigin-Name: 89377421ff69f2450364987afe781b6d8bcbf087
2014-08-01 11:16:25 +00:00
dan 84d6fb394f Fix the xColumnSize() extension API.
FossilOrigin-Name: 19504c4108472d2ad1281221642b8bd06eb69f4e
2014-07-21 11:44:47 +00:00
dan 48eecfb8b9 Support "ORDER BY rowid ASC".
FossilOrigin-Name: b96b5e166990e4ec363b24f66e04cfa5f00f6342
2014-07-10 20:21:12 +00:00