Commit Graph

1219 Commits

Author SHA1 Message Date
dan a90579db99 Fix two problems that could cause fts3 auxiliary functions to occasionally misbehave if used with match expressions that contain both OR and NEAR.
FossilOrigin-Name: 372c1db2475f367d54270d5801aff0503745bff4
2015-07-08 17:59:08 +00:00
dan bcf2163771 Merge the Fts5MultiSegIter and Fts5IndexIter objects.
FossilOrigin-Name: 6ace73541ae48aeb19e0c2d9f6772d032d0a7eae
2015-07-07 19:35:11 +00:00
dan 01233d3e54 Add a test case to verify that "PRAGMA data_version" works as expected when an OTA client writes to the database.
FossilOrigin-Name: 6441f2dc9e7f53de64bc15b1d766c885b2635112
2015-07-07 19:07:30 +00:00
dan b2fe306a2e Further optimizations for fts5 b-tree seeks.
FossilOrigin-Name: f37899686c032145f431f81c1072738536c61c7e
2015-07-07 08:29:32 +00:00
dan 656773b675 Speed up seek operations on fts5 b-tree structures.
FossilOrigin-Name: 7b7da1eb435d321fc4283f6aa2161fa1e16f2cf3
2015-07-06 20:27:19 +00:00
dan 30ae503167 Optimize seek operations on fts5 b-trees.
FossilOrigin-Name: 8cf02090ce53ec150492d77d9e5e5f27665bd34f
2015-07-04 18:44:07 +00:00
dan 70c8885417 Fix a problem in the fts5 integrity-check.
FossilOrigin-Name: 4e473559c9bdcf4d0b823e995a3642cd50ebd337
2015-07-04 15:35:02 +00:00
dan 28a10c3c1f Rework the Fts5MultiSegIter structure a bit to make it more efficient.
FossilOrigin-Name: 0778825d0ec9315c70659fae8d0640b209049dd8
2015-07-03 20:47:18 +00:00
dan 00a1351e55 Speed up eof checks on fts5 cursors.
FossilOrigin-Name: 3df4af5d8c28863783b0bc867abfbe31cc96f1b9
2015-07-03 19:13:56 +00:00
dan ec7c1de354 Avoid freeing the array of instance matches each time the fts5 xNext() method is called.
FossilOrigin-Name: b9b0c1e50d77f5d6e02f43fbb100c722cb692cc5
2015-07-03 17:14:18 +00:00
dan a153bbc4c2 Copy fts5.h into the build directory as part of building fts5.c. Also remove code to edit fts5parse.c from various Makefiles and add it to mkfts5c.tcl.
FossilOrigin-Name: 4d8f03dc7b6529cf8d46fc0fcc82112c0394dc13
2015-07-02 20:12:34 +00:00
dan 6544b8643f Minor change to mkfts5c.tcl so that fts5.c can be used according to the instructions in loadext.html.
FossilOrigin-Name: edbcccd349090efff3f975e27451608136a429fc
2015-07-02 18:52:16 +00:00
dan 3abdc44e7b Add fts5_source_id() SQL function, similar to sqlite_source_id().
FossilOrigin-Name: ec2573487cb86664f7f056168a764c28ca8236fc
2015-07-02 18:16:02 +00:00
dan 3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
2015-07-02 15:52:21 +00:00
dan ef648698c8 Fix minor problems in the ota demo application.
FossilOrigin-Name: 6aaaec6e63cdf713b0d67e24a892088ff251c82a
2015-06-29 15:41:50 +00:00
dan 018a93d216 Remove unnecessary lines from fts3.h and main.mk.
FossilOrigin-Name: c1b268ced38259706674ed2fcbf842d3fbd9fb8c
2015-06-26 20:13:56 +00:00
dan 6394d99a0e Fix a segfault that could follow an OOM error in fts5.
FossilOrigin-Name: 713239b8cf2900e8f7d97646c7f350248b4e804f
2015-06-26 20:08:25 +00:00
dan 6fbeb22596 Fix some cases in the fts5 code where a corrupt database could cause a buffer overread.
FossilOrigin-Name: 360c57bbb8bad6cc84af140f61e0282fa06462b4
2015-06-26 18:50:29 +00:00
mistachkin a468027c8d Merge upstream changes. Get the test fixture working with FTS5. Fix compiler warnings.
FossilOrigin-Name: bfcd16089cf8c65130c011a9718bf7812685c841
2015-06-26 16:55:55 +00:00
dan eb37567fe2 Remove fts5 initialization code from core file main.c.
FossilOrigin-Name: c91a93b343b4d3b1a19afdb5d23d1c508f9779d2
2015-06-26 16:42:17 +00:00
mistachkin ed52f9ff48 Initial changes to get FTS5 working with MSVC.
FossilOrigin-Name: ef2052f81e33ca98e85a60f8a78cdd19a7c1c35c
2015-06-26 04:34:36 +00:00
dan 3e65f89ef6 Fix Makefile.in to account for recent changes. Update fts5 test scripts so that they do not run if SQLITE_ENABLE_FTS5 is not defined.
FossilOrigin-Name: 3175220747f5d4dadd17e09fb1899dc782c90926
2015-06-25 20:36:36 +00:00
dan f32fa3116b Merge latest trunk changes into this branch.
FossilOrigin-Name: fcd8f7ce601729dc51d880d16b97040c1be16aa2
2015-06-25 20:16:23 +00:00
dan ed2589f555 Add a script to combine all fts5 code into a single file - fts5.c - that can be used to build an SQLite loadable extension.
FossilOrigin-Name: 46e86b0637248fb4d623c97778cc041eabe3636c
2015-06-25 20:10:24 +00:00
dan 88d702e6bd Add "ON CONFLICT" handling to the spellfix module.
FossilOrigin-Name: 1d04def785b6031de68b7f199d400cbb5c76caea
2015-06-24 17:21:52 +00:00
dan 51ef0f57c7 Improve test coverage of fts5.
FossilOrigin-Name: df5ccea80e8f0da83af5e595b539687006085120
2015-06-23 18:47:55 +00:00
dan c94a08100b Merge latest trunk changes with this branch. Add tests for columnsize=0.
FossilOrigin-Name: ef44c71a22518727030dd90c0139af8973b05841
2015-06-23 15:06:13 +00:00
drh f5ed7ad63e Fix harmless compiler warnings.
FossilOrigin-Name: b0badb99023e23bef0e2064cec58f2b279af0c0b
2015-06-15 14:43:25 +00:00
dan 1153e60cf0 Fix the fts5 xRename() method.
FossilOrigin-Name: 0f7fd51325875fbf0f1eaca3bbbd170ef99c4208
2015-06-10 10:45:34 +00:00
dan bcc2f04c68 Add the "columnsize=" option to fts5, similar to fts4's "matchinfo=fts3".
FossilOrigin-Name: aa12f9d9b79c2f523fd6b00e47bcb66dba09ce0c
2015-06-09 20:58:39 +00:00
dan 4f20cd402b Avoid passing constraints that are unusable due to LEFT or CROSS joins to virtual table xBestIndex() methods.
FossilOrigin-Name: 80ee56dda7db3860f8be5f6968c8745138f8453f
2015-06-08 18:05:54 +00:00
dan 90502c3dc7 Fix a comment in fts5.h.
FossilOrigin-Name: e964b5877497b16cf985d3d847e82529bb3fa4a3
2015-06-06 19:23:32 +00:00
dan 7193b69f52 Fix handling of fts5 rowid constraints in the absence of a MATCH clause. Add tests to cover recently added branches.
FossilOrigin-Name: 3a9cb648138a059862fb438c0787fdd888f5e88e
2015-06-06 16:28:29 +00:00
dan e4449454c5 Make use of range constraints on the rowid field of an fts5 table in full-text queries.
FossilOrigin-Name: 32cbc0ed3699cc21302f0b6a159493117ad4bd4f
2015-06-05 19:05:57 +00:00
dan 6f277eb6bc Fix an fts5 problem in extracting columns from position lists containing large varints.
FossilOrigin-Name: 4ea015ab983300d420ef104cca550b22a6395866
2015-06-03 11:23:30 +00:00
dan f67bb4a21b Change the fts5 multi-column syntax to use parenthesis instead of square brackets.
FossilOrigin-Name: ab85a6fc4f7580278fc9d1f0090fdcf0a90d065b
2015-06-02 19:38:15 +00:00
dan 167fd757f0 Merge latest trunk changes with this branch.
FossilOrigin-Name: c9ffda4abb4390bbc5719e269196e2807b254f97
2015-06-02 18:07:58 +00:00
dan dbef6e752e Reimplement [ec69e09a] so that each call to the xNext() method does not involve two iterations of the match expression tree (only one).
FossilOrigin-Name: 80fe305b3eefb17310a9d6185d1c8cd73ee38b1e
2015-06-02 17:57:01 +00:00
dan 5350d91ed3 Improve performance of the fts5 AND operator.
FossilOrigin-Name: b43e9a5b7a0483ccb102316a4dbc5e32b5bc69ec
2015-06-01 19:17:06 +00:00
dan 34efc82eed Change fts5 expression processing to avoid linear scans of long doclists caused by phrases that match specific columns only.
FossilOrigin-Name: ec69e09a55b4daf1c40aeaaf9ee95091fe86f5c0
2015-06-01 09:15:20 +00:00
dan 3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h.
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
2015-05-30 11:49:58 +00:00
dan a8c024905c Add extra tests and fixes for multi-column matches.
FossilOrigin-Name: ae6794ffa23ef6191bd8834422abf322d978c11b
2015-05-29 19:00:22 +00:00
dan 6d21f42db2 Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>".
FossilOrigin-Name: 0fc0ea20920615f3e48ea2dbe2b7dcd979b0993e
2015-05-29 15:55:30 +00:00
dan e96efb6426 Optimizations for fts5 queries that match against a specific column.
FossilOrigin-Name: b29ac50af0491a780a5a4c0985d88d0e5e014ba3
2015-05-28 19:57:12 +00:00
dan af5cd564f0 Remove some dead code from fts5. Add auxiliary function api tests to the same.
FossilOrigin-Name: 0f9df202cc58097afddb8dad662b7c7fdc2c7d0c
2015-05-28 14:37:26 +00:00
dan 7f64db1343 Simplifications and minor optimizations to fts5 prefix queries that cannot use a prefix index.
FossilOrigin-Name: aef89d9f6aa536efee347367558cf5d4ea81b238
2015-05-26 18:22:01 +00:00
dan add0804943 Avoid using a zero-sized array within a struct in fts3 code.
FossilOrigin-Name: b05cae36cedd98d59813e637f328a52eee7ef0d2
2015-05-26 17:29:48 +00:00
dan 494016a424 Avoid redundant loads from the %_data table in the fts5 code.
FossilOrigin-Name: 02069782f8b7896a582582c79185b50418622736
2015-05-25 11:46:33 +00:00
dan 3fb37f097c Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions.
FossilOrigin-Name: f1e942a1dda496a509741e9cc2a17e8b4dac63a3
2015-05-25 10:57:13 +00:00
dan 2e7d35e2fe Avoid making redundant copies of position-lists within the fts5 code.
FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0
2015-05-23 15:43:05 +00:00
drh 18070e08c5 Fix incorrect validation of the size of the BLOB returned by a geometry
function on the RHS of a MATCH operator in RTree.  The old code worked for
64-bit systems (by chance) but fails on 32-bit.

FossilOrigin-Name: 10cc44f5a63e6cadf6940bb7310a177ee7fa6ee2
2015-05-22 23:17:28 +00:00
drh fd29973f11 Update with fixes and enhancements from trunk.
FossilOrigin-Name: 9605d008f5c763137e9734d033fe4510ca316858
2015-05-22 18:40:48 +00:00
dan 6b76418e67 Have r-tree prefer to use the constraint "col MATCH ?" over "rowid = ?".
FossilOrigin-Name: b9fb95194d4f7c535f1d175fd2e18d69d76e8fc4
2015-05-22 16:08:42 +00:00
dan 0d45967b7f Increase test coverage of fts5_vocab.c.
FossilOrigin-Name: 065ab83a6ce36e16d3b95a61505aa3cff0bfea84
2015-05-22 07:44:44 +00:00
dan 21b7d2a9b8 Improve test coverage of fts5_unicode2.c.
FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e
2015-05-22 06:08:25 +00:00
drh 4f03f413bc Add the sqlite3_value_dup() and sqlite3_value_free() interfaces. Use
these interfaces to enhance R-Tree to add the
sqlite3_rtree_query_info.apSqlParam field.

FossilOrigin-Name: a7ee40c4fc62843ac5b96ba47ca14a66e8cd6961
2015-05-20 21:28:32 +00:00
dan 8c1f46de50 Improve test coverage of fts5_tokenize.c.
FossilOrigin-Name: 0e91a6a520f040b8902da6a1a4d9107dc66c0ea3
2015-05-20 09:27:51 +00:00
drh 7ac07bda73 Fix the MSVC makefile so that it works with OTA. Fix a harmless compiler
warning in FTS3.

FossilOrigin-Name: c7b16645307d6e46c4153693d6806269de64955f
2015-05-19 22:56:45 +00:00
drh 856d446efe Allow R-Tree geometry functions to take 8-byte BLOB arguments which are
passed directly through to the underlying callback, and which can be used
to pass pointers into the callback.

FossilOrigin-Name: b271ed56532a78323accc8a7cb348d55f95c350e
2015-05-19 22:20:48 +00:00
dan 116eaee4a0 Add tests for fts5 tokenizers.
FossilOrigin-Name: 4f90ba20e2be6ec5755fe894938ac97342d6fbf6
2015-05-19 19:37:09 +00:00
dan 815bd625c2 Remove the sqlite3ota_open_v2() API. Add a new parameter to sqlite3ota_open() instead.
FossilOrigin-Name: c74e0bc481ce995f83ca8384e05dfbe068a0ae85
2015-05-19 17:48:11 +00:00
dan a6fb464878 Allow OTA update state data to be stored in a database separate from the OTA update database.
FossilOrigin-Name: 5af8db56af457d60ea030d84666ca7fffb6821fe
2015-05-19 16:22:58 +00:00
dan b10210ea1b Fix a memory leak that could follow an OOM condition in fts5.
FossilOrigin-Name: de9f8ef6ebf036df5a558cd78fb4927da2d83ce8
2015-05-19 11:32:01 +00:00
dan 09963b27b8 Improve test coverage of fts5_storage.c.
FossilOrigin-Name: 4dfe2e5871704125338620705e6638f36baaacd7
2015-05-18 20:34:59 +00:00
dan 23390b1b2b Fix a bug in test script fts5fault4.test.
FossilOrigin-Name: a508059305641c2cb53197b0f6e93d7b723697e2
2015-05-18 18:24:41 +00:00
dan 5211e54f95 Add a test for an untested branch in fts5_expr.c.
FossilOrigin-Name: ce08206b5c5bf6a81decf20c99a524dd92c3715b
2015-05-18 18:03:06 +00:00
dan 27aac274b9 Improve test coverage of fts5_config.c.
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
2015-05-18 17:50:17 +00:00
dan e0569a4aae Further test coverage improvements for fts5.
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
2015-05-16 20:04:43 +00:00
dan 369e4129b9 Improve test coverage of fts5_index.c.
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
2015-05-15 18:13:14 +00:00
dan 415ae725cb Add test cases. And some fixes.
FossilOrigin-Name: adee788586197445672013d434e7ba47ce510b59
2015-05-15 12:18:39 +00:00
dan 8255079611 Merge latest trunk changes with this branch.
FossilOrigin-Name: 299ed55c909d327826ef47e204ef1e9afd54852b
2015-05-14 08:58:52 +00:00
dan 5d03b9d365 Merge latest trunk changes with this branch.
FossilOrigin-Name: b5f0e8c5b4bc018d672617ffd342d12b228548b9
2015-05-13 18:12:58 +00:00
dan 23d538885e Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token.
FossilOrigin-Name: aa34bf666c384cf32a8d8166ab6d9afbca26a256
2015-05-13 17:15:32 +00:00
dan 1c1cea8bae Merge latest trunk changes into this branch.
FossilOrigin-Name: 82e5a6e088c58815140ad36715ac11c96527cb25
2015-05-11 18:46:42 +00:00
dan 71ab324066 Allow the fts5vocab table to optionally provide data on a per-column basis.
FossilOrigin-Name: 3922276135a7825d0ede8d9c757e9cfe492f803a
2015-05-09 18:28:27 +00:00
dan 5e38f1c9bf Add the fts5vocab module, for direct access to the fts5 index.
FossilOrigin-Name: 6bf93e3b56e6705b7d12bab5024fc615f373b36c
2015-05-08 20:21:24 +00:00
dan 76724372ae Improve the error message returned by FTS5 if it encounters an unknown file format.
FossilOrigin-Name: f369caec145f311bb136cf7af144e2695badcb9b
2015-05-08 09:21:05 +00:00
mistachkin 6a12854024 Remove a couple stray test breakpoint calls.
FossilOrigin-Name: 2860cebeeaebd346de60c762aa3e51dbab008578
2015-05-08 00:58:39 +00:00
dan 4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
2015-05-07 19:29:46 +00:00
dan 739b903e6d Remove some dead code from fts3_snippet.c.
FossilOrigin-Name: 46b2d3cef5c22a9e6bd0a4f8411f17b7ec72bd18
2015-05-06 17:51:59 +00:00
dan e60aedc564 More optimization for the 'y' and 'b' matchinfo directives.
FossilOrigin-Name: 8c5b9fedfcee3ac22a222819dceb981ad94a9903
2015-05-06 17:41:19 +00:00
dan 753c1f56db Further optimizations for the 'y' and 'b' matchinfo operators.
FossilOrigin-Name: fbd038bb57322e1ed2e1ee52f3d134594b6bfcc0
2015-05-06 08:43:26 +00:00
dan 99eaf39784 Add the fts3 matchinfo 'b' flag.
FossilOrigin-Name: b9b77972d88171e4239b8194f308eb5d60b5d172
2015-05-05 20:39:53 +00:00
dan b2bd727e52 Optimizations for the matchinfo() function, particularly the 'y' flag.
FossilOrigin-Name: dddd7e182943a1d3a9d32830e819a63f1a228d6d
2015-05-05 19:37:07 +00:00
dan 7c479d51e5 Reorganize some of the fts5 expression parsing code. Improve test coverage of the same.
FossilOrigin-Name: c4456dc5f5f8f45f04e3bbae53b6bcc209fc27d5
2015-05-02 20:35:24 +00:00
dan 28f98455f4 Add the experimental matchinfo 'y' flag to fts3/4.
FossilOrigin-Name: 92941609af74044b3078e020324a37b04a0638b0
2015-05-02 09:44:15 +00:00
dan 0501b2492a Further improvements to test coverage of fts5 code.
FossilOrigin-Name: d4331943dff259380c4025bb740d8aba6972d351
2015-05-01 20:38:57 +00:00
drh e9c5f97683 Fix an assert in RTREE that would fire if the rtree table is misdeclared.
FossilOrigin-Name: 9a45409cc4078f2b6e68aa777f6ab86a14309833
2015-05-01 18:00:37 +00:00
drh abf582509d Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3,
and to ensure that error messages are not overwritten and thus leaked.

FossilOrigin-Name: 605347e087ec5eb817f3c94f4616abe26c7ab66b
2015-05-01 14:07:30 +00:00
dan 557085a5a1 Improve test coverage of fts5.c.
FossilOrigin-Name: add4f4681c648dcbecaa68d08f7b2f4e6d63003c
2015-05-01 12:14:23 +00:00
dan 7b2ec1ae41 Improve fts5 tests.
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
2015-04-29 20:54:08 +00:00
dan 90dd70226f Fix an fts5 bug in handling writes while there are active cursors.
FossilOrigin-Name: 07f70955392697556ca2951c9b6c3a5204cd5ec0
2015-04-28 20:24:50 +00:00
dan cdbd150170 Improve coverage of fts5 tests.
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
2015-04-28 18:35:28 +00:00
dan a3bdec7ee4 Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement.
FossilOrigin-Name: e38e2bb637844dae8ae5d5f3e23d8369e1b91e45
2015-04-27 16:21:49 +00:00
dan c1cea8f731 Further tests for fts5.
FossilOrigin-Name: ffeb3ef3cfec3681b72bb28cfa612aa15e07887d
2015-04-27 11:31:56 +00:00
dan 26c84373c4 Improve coverage of fts5_index.c slightly.
FossilOrigin-Name: e5aaa01306597ffd2475dcb83ae889393f68d315
2015-04-25 20:29:46 +00:00
dan b33e64c83e Add tests for fts5.
FossilOrigin-Name: e748651c940eae2389fe826cf5c25f1166a5e611
2015-04-25 18:56:48 +00:00
dan ef8b74324d Merge latest trunk changes with this branch.
FossilOrigin-Name: 1c78d8920fb59da3cb97dd2eb09b3e08dfd14259
2015-04-24 20:18:21 +00:00
dan df5bd1fed2 Add the "unindexed" column option to fts5.
FossilOrigin-Name: 86309961344f4076ddcf55d730d3600ec3b6e45c
2015-04-24 19:41:43 +00:00
dan def90aae18 Add extra tests for corrupt database handling in fts5.
FossilOrigin-Name: 41449f7a0b5da6332eef48386c91ef63382c4783
2015-04-24 15:56:09 +00:00
dan 6758959ec3 Fix a problem in the "ota" utility program causing it to crash if SQLite failed to open the target database.
FossilOrigin-Name: a1a2b4a2eff4f641fa80e8ffa805c97afb73be95
2015-04-23 19:38:20 +00:00
dan e11afed562 Merge latest trunk changes into this branch.
FossilOrigin-Name: 1506e99eca19f72fb59b55ec9e2534505dfa7a6e
2015-04-23 19:32:19 +00:00
dan 55d46f01c5 Fix a performance problem in calls to sqlite3ota_close() made after the OTA update has been completely applied and checkpointed.
FossilOrigin-Name: fa62093b2531424846ea500c4155170d1bc7110e
2015-04-23 19:18:42 +00:00
dan 1cdd523a50 Have OTA maintain a small LRU cache of UPDATE statements. This reduces the amount of time it spends compiling UPDATE if a single data_xxx table contains many different ota_control strings.
FossilOrigin-Name: baee3556ea10d96f1623cf4dce112fa1a1070820
2015-04-23 18:14:21 +00:00
dan 38f2024062 Fix a problem causing the fts3 integrity-check to fail if run inside a transaction.
FossilOrigin-Name: 3b925189a75eae875da256b6e54999cad3cef1eb
2015-04-23 11:52:04 +00:00
dan 206480992d Add extra OOM tests for fts5.
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
2015-04-22 20:58:24 +00:00
dan c51f5b3fd9 Fix problems with doclist-indexes involving very large rowids.
FossilOrigin-Name: 1e5994097e4c740c5173ea9718c3935728fdb86f
2015-04-22 20:14:46 +00:00
dan 39b7ff7987 Add comments related to database locking to sqlite3ota.h.
FossilOrigin-Name: 77242965e77446313a8f0a65a39fccb67fe4cabf
2015-04-22 11:34:38 +00:00
dan 062c267dff Update this branch with latest trunk changes.
FossilOrigin-Name: 9797482ded7de985e3b20aedec5e4d81f55065c8
2015-04-22 09:40:35 +00:00
dan 5bb97c2b80 Fix a problem in fts5fault1.test.
FossilOrigin-Name: a21d60cb2ac6463c012d82d1970d90da5da2a14a
2015-04-21 20:13:38 +00:00
dan d7889b26ac Fix an fts5 problem with large deletes.
FossilOrigin-Name: e50e8031d6f804ebe50e0eec9a8b6e7f0152ecc3
2015-04-21 19:07:39 +00:00
dan 903dd7d5d1 Add a comment to sqlite3ota.h. No code changes.
FossilOrigin-Name: 352fa09efd8240567b8f9487944e8c3ade4e664a
2015-04-21 15:35:30 +00:00
dan 3c9a073723 Fix an fts4 problem to do with the handling of phrases that contain zero tokens.
FossilOrigin-Name: caba5be0b2bdaf4eb4af9af267c0c22dfbd3dba5
2015-04-21 12:06:53 +00:00
dan 80d3ea080a Fix some fts5 problems with very large position lists.
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
2015-04-20 18:48:57 +00:00
mistachkin 8e18922f12 Fix harmless compiler warnings seen with MSVC 2015.
FossilOrigin-Name: d05becd873a03a366843a34e7f4c732dd8f88c86
2015-04-19 21:43:16 +00:00
dan 1586699cfc Fix a problem affecting some fts3 UPDATE and DELETE statements on order=DESC tables.
FossilOrigin-Name: 284c1623f81704ef80edb9324954525cb2b72172
2015-04-17 20:51:55 +00:00
dan 1c29c27bb0 Ensure that filenames passed to the VFS xOpen() function are terminated by 2 '\0' bytes.
FossilOrigin-Name: 66eff473c3634edb42f6ad6955acfba3e3a10def
2015-04-17 17:00:52 +00:00
dan 43030868e6 Fix a memory leak in sqlite3ota.c that can follow an OOM error.
FossilOrigin-Name: c3dc15e71782fc70e92b71ad9eec0cf8ffef611f
2015-04-17 16:29:11 +00:00
dan ec16d98489 Have OTA always specify SQLITE_CONFIG_URI when opening databases. Fix a test issue causing otacrash.test to fail.
FossilOrigin-Name: 0d0e5ec064eaecb200b9b601b7a54a1700cd176e
2015-04-17 08:36:05 +00:00
drh a4bb7764fe Fix a boundry-value condition in the phrase poslist extractor of FTS3/4.
FossilOrigin-Name: 55d10baf0bffdb1a34bf5627ed8f25e4a4efd942
2015-04-16 20:10:17 +00:00
dan 5c06dd831f Be sure to release any xShmLock locks held when closing an OTA handle.
FossilOrigin-Name: d0fba72a47f73082ade40a732aab114bc8f6d83e
2015-04-16 18:49:53 +00:00
dan 28b22a0159 Fix a mismatched printf() argument and format specifier. Add ota files to Makefile.in.
FossilOrigin-Name: 5db810a88dac5cbf1f6b321345d01cc0d9d05d31
2015-04-16 14:33:09 +00:00
dan 4141a81b30 Avoid attempting to rename the *-oal file while it is still open.
FossilOrigin-Name: 658c3b0c80b9fe7eaac5982573651dc649a4c53f
2015-04-16 10:55:58 +00:00
dan 687c5124e0 Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances.
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
2015-04-15 18:49:20 +00:00
dan dbca0ab9b0 Fix a problem preventing doclist indexes from being loaded.
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
2015-04-15 16:01:42 +00:00
drh fdb031891c Merge all recent trunk enhancements and fixes into the ota-update branch.
FossilOrigin-Name: 9bd3e4453d4ad416f7e3f08f0bd283d34f1c319c
2015-04-15 14:26:04 +00:00
dan 0d01ec8de8 Remove an incorrect assert() statement from sqlite3Fts3Dequote().
FossilOrigin-Name: bd06eeb8d06237dc2d54d8a03e8bf525cb811c9e
2015-04-15 09:16:39 +00:00
dan 2deb12be2b Fix a problem causing an assert() to fail if a snippet containing 0 tokens was requested from fts3.
FossilOrigin-Name: eafd0a1e3f25f38d551603f177ff4634cf79de77
2015-04-15 08:20:50 +00:00
dan f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly.
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
2015-04-14 20:15:41 +00:00
dan 929b2cfc2e Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index.
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
2015-04-11 18:25:04 +00:00
dan 56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
dan 75395ccd17 Do not allow virtual table constructors to be called recursively.
FossilOrigin-Name: 0a72726da21581ab16cb3e964bd825b8f2e931e4
2015-04-10 07:55:07 +00:00
drh 57d0d9f33c Merge all changes for version 3.8.9 into the ota-update branch.
FossilOrigin-Name: ec9d907a57fcea72c8a64e521e8a1b5777d67568
2015-04-08 14:01:07 +00:00
dan 4b0000c9a8 Fix a problem with fts3 prefix terms within phrase queries on "order=DESC" tables with a mix of negative and positive rowids.
FossilOrigin-Name: 0cdf502885ea7e5805d7ba3719f055f5d48fc78d
2015-04-06 09:05:29 +00:00
drh f3e8748683 Fix a harmless compiler warning in FTS3.
FossilOrigin-Name: 1688f60c08cc9270948402d98a2333382aa32681
2015-03-26 14:29:51 +00:00
dan 6da7a0a93d Merge the latest trunk changes into this branch.
FossilOrigin-Name: 9d9b6c883b4f7d69c615cedfb59a2385aac47b74
2015-03-24 18:21:41 +00:00
dan f37120a416 When the OTA module updates or deletes a row, save only those fields that are part of an index or primary key to the ota update database.
FossilOrigin-Name: 6326fd3249bee267da0172f8afd1e7b3f71521b9
2015-03-24 18:03:29 +00:00
dan 5e4eeb3c00 Fix a broken assert() in the ota module.
FossilOrigin-Name: 858de8a5e7925bd96d7b9c0f00bc16358cf3b295
2015-03-23 17:10:51 +00:00
drh 3858cb44b6 Add a missing "int" on a constant declaration.
FossilOrigin-Name: 235157de5113ac4c750e36a498e2a1f1cf461751
2015-03-21 20:50:58 +00:00
dan e10d87f65a Avoid a segfault if NULL is passed as the first argument to SQL scalar function fts3_tokenizer().
FossilOrigin-Name: 6d0989695b486275824c14d5f88357267c1e8104
2015-03-21 19:35:09 +00:00
dan 71138b0f0f Merge trunk changes with this branch.
FossilOrigin-Name: 142743918fd5a6d79fa10c44398360c3684a255a
2015-03-21 15:45:24 +00:00
dan 8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
2015-03-21 15:37:19 +00:00
dan 998aaa03ea Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test.
FossilOrigin-Name: e3e234649616f20610abce9ae9da1c572d3a4377
2015-03-21 12:22:51 +00:00
mistachkin 44723ce096 Improvements to the MSVC build. Fix harmless compiler warnings. Enable use of 'stdcall'.
FossilOrigin-Name: 737630b87314283b2c38790ace9d25ec05f81f4a
2015-03-21 02:22:37 +00:00
dan b7134d0863 Fix an FTS3/4 problem with handling empty tokenizer declarations (e.g. "CREATE VIRTUAL TABLE t(tokenize=);").
FossilOrigin-Name: 26d2def8a53094356008861636d66f9ae8f2448a
2015-03-19 16:25:42 +00:00
dan e0417626ea Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer.
FossilOrigin-Name: ad4b19d2ac0889a23fe3b0fd844286efc10cdd82
2015-03-19 15:52:07 +00:00
dan c3b328da6e Allow whereShortCut() to use the PRIMARY KEY index of a WITHOUT ROWID table to optimize a vector of "IS" operators in a WHERE clause.
FossilOrigin-Name: 52e73eeca063bb30092ce600068bf487641399a0
2015-03-18 20:03:27 +00:00
dan 9340de52e2 Clarify that OTA is unable to update or delete rows with NULL values in primary key fields.
FossilOrigin-Name: 2e7c1e0a0d128d9bac119692b2505b5ed0abe87a
2015-03-18 19:04:40 +00:00
dan f5fab92d82 Add an optimization to the fts5 unicode tokenizer code.
FossilOrigin-Name: f5db489250029678fce845dfb2b1109fde46bea5
2015-03-11 14:51:39 +00:00
dan c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees.
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
2015-03-10 19:24:30 +00:00
drh a9106ca83d Merge recent trunk enhancements into the ota-update branch.
FossilOrigin-Name: 5489cb68921f62f10d832adbc4d19ea8c6c5da50
2015-03-09 13:24:46 +00:00
dan 92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions.
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
dan 6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
dan 54fc21438e Fix some problems with OTA and empty target databases, or target databases with the wrong set of tables. Also add SQLITE_FCNTL_VFSNAME support to the OTA VFS.
FossilOrigin-Name: 46119e8d8e391d8dea844352521b58415f7365b1
2015-03-05 16:21:20 +00:00
dan 2bd7ace54e Add comments to sqlite3ota.h to make it clear that passing NULL in place of a parent VFS name to sqlite3ota_create_vfs() causes the new VFS to use the system default as its parent.
FossilOrigin-Name: 158c1a48818a9abc001b9ea547167c2624a7bad3
2015-03-05 14:07:25 +00:00
dan 67d4b20c46 Fix a typo in sqlite3ota.h.
FossilOrigin-Name: 04087dec4c3db5f322eca289585525b7267ed4f8
2015-03-05 13:43:21 +00:00
mistachkin 0b7ff5c11b Fix harmless compiler warning.
FossilOrigin-Name: 580dae4615353d73c7d096107571bd60fec5b0f2
2015-03-04 20:18:55 +00:00
dan 47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan f400282bdf Further minor optimizations to flushing fts5 data to disk.
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
2015-02-27 09:41:10 +00:00
dan 5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
mistachkin 38e40ee2e5 Fix potential memory leaks in the misc 'compress' extension.
FossilOrigin-Name: 3bc34fd427d9d7819cd9740237b1f5d4180341fa
2015-02-26 21:04:44 +00:00
dan dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments.
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
dan 4a7e11c629 Fix an fts5 bug in large incremental merges.
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
2015-02-26 14:54:03 +00:00
dan 6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
dan 5b828f8c33 Fix building with ENABLE_OTA and OMIT_LOAD_EXTENSION.
FossilOrigin-Name: d6d21ff55228d3ffe6cdbb5755645b147806a1da
2015-02-23 20:09:50 +00:00
dan 04f121c187 Change SQLITE_FCNTL_ZIPVFS_PAGER to SQLITE_FCNTL_ZIPVFS.
FossilOrigin-Name: f7865b942834dd2f6b865336e08ba1adbf20612a
2015-02-23 15:41:48 +00:00
dan 5ebc83b1d2 Update the sqlite3ota_db() API to account for the fact that each OTA handle now uses two SQLite database handles.
FossilOrigin-Name: ef08ecceb7e237a01af6cc3141dccee09ffe9ae3
2015-02-23 15:02:13 +00:00
dan 882b8e0974 Improve tests for resuming ota updates following power failures. Fix a problem revealed by the same.
FossilOrigin-Name: 1cb675e5392f179516d8e7a52760922a6c7df4d0
2015-02-23 12:22:55 +00:00
dan ee9fa4af15 Fix some problems with resuming ota updates if saving the state is interrupted by a power failure or system crash.
FossilOrigin-Name: 6d5ed70d0dbabe9c2ab2f2dba47747d17d937781
2015-02-21 20:08:25 +00:00
dan 92f55ceb70 Add some missing comments and fix other minor code issues in sqlite3ota.c.
FossilOrigin-Name: 718fd8b673d6557dd0eaad03e6a3332b5490afbf
2015-02-20 14:36:16 +00:00
dan 942b0a68ea Ensure the mutex used to protect the linked list of all main database files opened by a single ota vfs is allocated.
FossilOrigin-Name: 9c8682d6650a94e11f9bec5baff69ed9668874fa
2015-02-19 19:59:35 +00:00
dan 138bf3bc7f Update various documentation comments in sqlite3ota.c and sqlite3ota.h.
FossilOrigin-Name: 60e0a46b82dd9c704e8aa977d1ccdd73d388422f
2015-02-19 18:06:40 +00:00
dan 20ab217f93 Add tests for a couple of previously untested branches in the ota code.
FossilOrigin-Name: a3c1bc5d5e3f4b197f48cbbc240608e94bfc2b45
2015-02-19 13:36:02 +00:00
dan 7551c5c3e6 Add new file ota12.test, containing tests for applying ota updates to live databases with other active reader/writer clients.
FossilOrigin-Name: 0864d127fe42fc0db7ab30a3ebf74c0114095648
2015-02-18 20:17:14 +00:00
dan e3e031ce85 Add ota tests to increase code coverage. Fix some minor issues in error handling within the ota code.
FossilOrigin-Name: 2b10c5d2b8b8b535d3dec0c68a777db16268e1e5
2015-02-18 20:16:15 +00:00
dan d092df51de Fix a problem with OTA updates in the presence of database readers.
FossilOrigin-Name: 144bb29ffcbfe96dc10c0224113e73a80e89314b
2015-02-18 17:40:05 +00:00
dan eadc772beb Improve test coverage of ota code a bit.
FossilOrigin-Name: a438fa6c9ad2fb1d78ac747172d07455d6381387
2015-02-17 20:49:42 +00:00
dan f5fe23909d Add extra tests and fixes for ota.
FossilOrigin-Name: e0b7151962fedbcac975f2216fd6b33b995a8945
2015-02-16 21:13:19 +00:00
dan 5902352fca Add further tests and fixes for ota.
FossilOrigin-Name: 62dc1fffc38cb157c15105098749b6dd0198eb84
2015-02-16 11:48:34 +00:00
dan c76c64f931 Move tcl test code from sqlite3ota.c to new file ext/ota/test_ota.c.
FossilOrigin-Name: f20779a6e890ba73bfaa904cefcf3a029b01fed4
2015-02-16 06:27:37 +00:00
dan 52800df9fc Change the way the "incremental checkpoint" function of OTA works in order to reduce the effect on the SQLite core code.
FossilOrigin-Name: b64a11a754dc56f3406d3b703531ebe9e4af4908
2015-02-14 18:58:22 +00:00
dan 01325e86e6 Merge the ota-update-no-pager_ota_mode branch into this one.
FossilOrigin-Name: 71887cd9b38def398d48eaf0ec34eeac3c7c5177
2015-02-11 17:05:17 +00:00
dan a3cf4f7ccb Merge latest trunk changes with this branch.
FossilOrigin-Name: 0b63e8dcbaec5043e353734e684c2a46552a3409
2015-02-11 16:54:48 +00:00
dan 2e24971f2a Ensure that an error is reported if an attempt is made to update a wal mode database via ota.
FossilOrigin-Name: 6fc5d4d26a603b3906f02ceea0f507780d0c35eb
2015-02-11 16:25:27 +00:00
dan cd73244144 Further tweaks to work with zipvfs.
FossilOrigin-Name: 0f152416be792457c52417aeb531ac860d12a5bd
2015-02-10 20:00:38 +00:00
dan 07427d0350 Add documentation and test cases for sqlite3ota_create_vfs(). Also code to detect errors in zipvfs/ota setup.
FossilOrigin-Name: e729668168f00325459bc2e9b515aa95e57f2754
2015-02-10 17:08:17 +00:00
dan faa8c4fdcb Add the sqlite3ota_create_vfs() and sqlite3ota_destroy_vfs() functions.
FossilOrigin-Name: 96443ecb6909141aa621a16e628455857d036482
2015-02-09 20:07:35 +00:00
dan de1f6a270b Add comments to explain the role of the ota vfs.
FossilOrigin-Name: 7bb633639d1e41067903a49653f09a823054e213
2015-02-07 20:20:32 +00:00
dan dd0ad3785b Remove "PRAGMA pager_ota_mode".
FossilOrigin-Name: 8ac58e46782bd6b81c06fdf1cb5b316b8a4e1ddf
2015-02-07 19:17:36 +00:00
drh 2f8109cef1 Revise the way that the index structure for a WITHOUT ROWID table is
discovered.

FossilOrigin-Name: 7f10a0eaf1fedfa020cbd7019ec9342ffdc3b9b0
2015-02-06 00:31:45 +00:00
dan 8800b53bc5 Change a comment in sqlite3ota.h to make it clear that it is not possible to insert a NULL value into an INTEGER PRIMARY KEY column using ota.
FossilOrigin-Name: a5e86bea4ad51bbb988a2a2215961706894f4a02
2015-02-05 17:46:19 +00:00
dan a8ccf3ed74 Prevent ota updates from violating NOT NULL constraints. Add a comment to the "limitations" section of sqlite3ota.h saying that CHECK constraints are not enforced.
FossilOrigin-Name: 74e073dd604142212f3d3e1931065d124daabd80
2015-02-05 17:36:30 +00:00
drh d4ef7c50bb Figure out the primary-key type of a table using queries of sqlite_master
and the table_info and index_list pragmas, obviating the need for
SQLITE_TESTCTRL_TBLTYPE.

FossilOrigin-Name: 50ecdfc443b51e3569c6add2fba5132f959c61cb
2015-02-05 01:49:31 +00:00
dan 06c9722564 Fix the error messages returned by ota if an update violates a unique constraint.
FossilOrigin-Name: c4845a3b591723ff60b7b1da198f44908c12bdfc
2015-02-04 19:20:42 +00:00
dan 738a3b2fcb Avoid a linear scan of the entire table when ota updates or deletes a row from a table with an external primary key index.
FossilOrigin-Name: 1db198ccca1e5c5a922cefe3daeff8d2e5d3a7f7
2015-02-04 16:32:47 +00:00
dan 1e35850229 Fix a memory leak that could follow an OOM error in ota.
FossilOrigin-Name: 0d5415f26427d0af89663c5a3e944e488143b648
2015-02-04 11:08:47 +00:00
dan e2863b6074 Fix some problems surrounding WITHOUT ROWID tables with DESC primary key indexes
.

FossilOrigin-Name: a21fefb79a161e6bb057ae4f6ba554f659706ac1
2015-02-03 18:43:42 +00:00
dan 75fda9b395 Remove "PRAGMA ota_mode".
FossilOrigin-Name: 1c111447a07687c30ed4ad5a6c27a169c85b7ea6
2015-02-03 15:56:08 +00:00
dan 57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation.
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
dan 32348f7bd2 Merge latest trunk changes with this branch.
FossilOrigin-Name: 76212f2c9a3c3ff0e238d6dad776938c6af674e6
2015-02-02 09:40:36 +00:00
dan 3b660d7da3 Have ota use imposter tables to write to indexes instead of the sqlite3_index_writer() interface. The error handling in this version is broken in a few small ways.
FossilOrigin-Name: cdaeab467f6aa3217be161377a9b78a4eec37093
2015-01-31 20:42:04 +00:00
dan 5082140311 Minor optimizations to fts5 writes.
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
drh 3102aa01fd Merge all recent trunk changes, and especially the SQLITE_TESTCTRL_INITMODE
enhancement.

FossilOrigin-Name: 36436dde74ce2536a9a430b4458216499ad113bf
2015-01-30 16:36:17 +00:00
dan 719b0f66e5 Fix some problems with transactions that both read and write an fts5 table.
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
dan 88392bf3ce Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query.
FossilOrigin-Name: 85dc12625d300fe48f3c096f54ebcb8b6ef4e30a
2015-01-29 11:52:22 +00:00
drh 120b3a4f61 Merge in all changes from trunk.
FossilOrigin-Name: 17c69be80542c5f84e21d60df3edc49422b087d9
2015-01-28 12:00:40 +00:00
mistachkin e4a0d79b8a Fix harmless compiler warnings.
FossilOrigin-Name: e7d2ec048c88237c124fbe598f8f7e950d43d90f
2015-01-27 21:24:33 +00:00
dan 900f1922ac Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
2015-01-27 20:41:00 +00:00
dan 6f0138e89e Fix a bug in the fts3 snippet() function causing it to omit leading separator characters from snippets that begin with the first token in a column.
FossilOrigin-Name: adc9283dd9bc3a6463f8c4fe23dd58a3712c349d
2015-01-27 19:01:26 +00:00
dan 18f6ff9eb7 Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function.
FossilOrigin-Name: 245e8730451fbdc1c729beff7295c452df604009
2015-01-27 18:43:02 +00:00
dan 8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan 641cb4360a Fix compression of keys stored on internal segment b-tree nodes by fts5.
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
2015-01-23 17:43:21 +00:00
dan 626d9e3062 Remove some redundant code from fts5.
FossilOrigin-Name: 939b7a5de25e064bdf08e03864c35ab718da6f6f
2015-01-23 06:50:33 +00:00
dan 0b520cc5f9 Add further tests and fixes for fts5.
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
2015-01-22 19:13:08 +00:00
dan 4c2871bead Further tests and fixes for fts5.
FossilOrigin-Name: c020a291ed293a66d21c5885e50a7fee04aa6366
2015-01-21 20:30:14 +00:00
dan dd8a501019 Fix an fts5 issue with loading doclist-indexes for a term that is the last thing on its leaf page.
FossilOrigin-Name: e0d614425f7f5ffe266fdc03642931b1ec19ad25
2015-01-21 18:23:25 +00:00
dan b461380333 Add further tests for fts5 backend.
FossilOrigin-Name: 09dabb3b9e140eec6cfda83bcb86b6b9f5cf54b6
2015-01-21 16:10:59 +00:00
dan 997d798d92 Merge latest trunk changes with this branch.
FossilOrigin-Name: b3348b1e07e168b156636a29fc8c6d6afb3129c2
2015-01-21 06:36:07 +00:00
dan fd0b436e10 Add extra fault injection tests to fts5.
FossilOrigin-Name: f45a0dc0a8911c8aac5a1028ac4f543a709656e7
2015-01-20 20:34:17 +00:00
dan aa4d380a42 Handle the case where a tokenizer determines that there are zero tokens in an fts5 query term.
FossilOrigin-Name: 75f3d17f864072dfa2caee182b86cc4b9972d691
2015-01-19 11:15:36 +00:00
drh 65545b5993 Eliminate all use of sprintf(), strcpy() and strcat() from test logic because
OpenBSD hates those functions.

FossilOrigin-Name: 10321910990195878c0af1e94b34ae0cdc0cb31b
2015-01-19 00:35:53 +00:00
dan d8736bc3b8 Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration.
FossilOrigin-Name: f30afd209aa4ce42766b1493750c4f5b5f1e9502
2015-01-17 20:01:52 +00:00
dan 2656167f6e Improve the performance of the fts5 porter tokenizer implementation.
FossilOrigin-Name: 96ea600440de05ee663e71c3f0d0de2c64108bf9
2015-01-17 17:48:10 +00:00
drh 068a251d99 Simplify some code in rtree, to avoid confusing the optimizer in GCC on
some macs:
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00).
Prior to these changes, compiling with -O3 would cause incorrect code to
be generated.  The change to the nodeGetCell() routine is key.  The other
changes are merely cosmetic details discovered while bug hunting.

FossilOrigin-Name: 882181ff9dd75f32db266db6e476671021fc567b
2015-01-13 21:26:17 +00:00
dan 851ca6e715 Fix prefix indexes so that they work in characters, not bytes.
FossilOrigin-Name: af8d43a4a08528bbae25ee38fe25de8a86f8a21c
2015-01-13 17:25:08 +00:00
mistachkin 7bb22ac7ff Silence some harmless uninitialized local variable warnings.
FossilOrigin-Name: 44375a34e59d1b0f0d765a99dcc7e6f30a4be7f7
2015-01-12 19:59:12 +00:00
mistachkin 7bb6e8e169 Get things compiling cleanly with MSVC and W4.
FossilOrigin-Name: c8725fa5fa361959b0f0a3fb36f204905d6f0ae9
2015-01-12 18:52:41 +00:00
mistachkin 1a51ce78ba Fix some harmless compiler warnings.
FossilOrigin-Name: 2b8eecbfe7935d3d9826fbcd473dd3a49138ad11
2015-01-12 18:38:02 +00:00
dan 73f7d6ed75 Optimize the unicode61 tokenizer so that it handles ascii text faster. Make it the default tokenizer. Change the name of the simple tokenizer to "ascii".
FossilOrigin-Name: f22dbccad9499624880ddd48df1b07fb42b1ad66
2015-01-12 17:58:04 +00:00
dan 27277c4e3c Fix some documentation issues in fts5.
FossilOrigin-Name: 512e1bdb4093b59d1494dfc63391476eadd52aea
2015-01-10 20:34:27 +00:00
dan d6b1c880d2 Add the fts5 'optimize' command.
FossilOrigin-Name: e749be563d8e738af113bd301770e2f22763ab77
2015-01-07 19:33:11 +00:00
dan 96a32bb51c Add the 'rebuild' and 'delete-all' commands.
FossilOrigin-Name: 0cb2fed525778d96237b5b0943047665e1f636d1
2015-01-07 17:11:11 +00:00
dan aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
2015-01-06 19:08:26 +00:00
dan 2a28e507f7 Further fixes and test cases related to external content tables.
FossilOrigin-Name: ce6a899baff7265a60c880098a9a57ea352b5415
2015-01-06 14:38:34 +00:00
dan ded4f41d1a Tests and fixes for fts5 external content tables.
FossilOrigin-Name: 047aaf830d1e72f0fdad3832a0b617e769d66468
2015-01-05 20:41:39 +00:00
dan 0fbc269fef Add support for external content tables to fts5.
FossilOrigin-Name: 17ef5b59f789e9fa35c4f053246d819987fd06f8
2015-01-03 20:44:58 +00:00
dan ade921c3ad Allow the rank column to be remapped on a per-query basis by including a term similar to "rank match 'bm25(10,2)'" in a where clause.
FossilOrigin-Name: 1cd15a1759004d5d321056905dbb6acff20dc7d9
2015-01-02 14:55:22 +00:00
drh 4081d5da3b Add logic to releasetest.tcl that detects -fsanitize=undefined errors.
Fix a few false-positivies that come up when running the sanitize=undefined
test.

FossilOrigin-Name: 2835e79a0afec6e4d449ac9340afec068c2d4c11
2015-01-01 23:02:01 +00:00
dan 37db72f1f7 Merge latest trunk changes with this branch.
FossilOrigin-Name: 4b3651677e7132c4c45605bc1f216fc08ef31198
2015-01-01 18:03:49 +00:00
drh f0a88279bd Fix a harmless compiler warning in rtree.
FossilOrigin-Name: 30891c6b8ebe9dfc939f9695bb45a159fbaaf262
2015-01-01 16:47:43 +00:00
dan 6024772ba2 Add a version of the unicode61 tokenizer to fts5.
FossilOrigin-Name: d09f7800cf14f73ea86d037107ef80295b2c173a
2015-01-01 16:46:10 +00:00
dan e716aca24b Move all fts5 test files to new directory "ext/fts5/test".
FossilOrigin-Name: 7f148edb30103c5f4fee20cd08e38537f9615bf2
2014-12-29 15:59:36 +00:00
dan 5fa3acabf4 Fixes to built-in tokenizers.
FossilOrigin-Name: b33fe0dd89f3180c209fa1f9e75d0a7acab12b8e
2014-12-29 11:24:46 +00:00
dan 2a615fa627 Fix the fts5 bm25() function so that it matches the documentation.
FossilOrigin-Name: 1ac7a8d0af9a71ddf6a1421033dcb9fa67c6120c
2014-12-23 19:18:34 +00:00
dan 005e10e39a Fixes and simplifications for the snippet() and highlight() functions.
FossilOrigin-Name: ca5d44042aa7461dcc8b700b0763df4df9d4a891
2014-12-22 21:01:52 +00:00
dan 1616d55153 Remove the fts5_test() aux function. Test aux functions using the tcl interface instead.
FossilOrigin-Name: 67e3ffd950c5347d219a06b33ad51949cffa7d90
2014-12-19 20:53:51 +00:00
dan 2ae0f4c931 Fix a problem with prefix queries and the AND operator.
FossilOrigin-Name: 38b3c65e3ee95eb7afadb76e0110570fbbc41e1b
2014-12-18 20:01:15 +00:00
dan e4bec37900 Fix various problems in fts5 revealed by fault-injection tests.
FossilOrigin-Name: e358c3de5c916f2c851ab9324ceaae4e4e7a0fbd
2014-12-18 18:25:48 +00:00
dan 0a99e9fb01 Update this branch with latest trunk changes.
FossilOrigin-Name: 69a312ad3fe5b39bc394b9ce958cb63d734518c7
2014-12-08 07:50:31 +00:00
dan fdf3a8970d Update comments in sqlite3ota.h to remove the "must have PRIMARY KEY" restriction.
FossilOrigin-Name: 088a41eb8c18886a260cf53fa0cca3bd1958dc05
2014-12-08 07:28:26 +00:00