Commit Graph

1219 Commits

Author SHA1 Message Date
drh 18070e08c5 Fix incorrect validation of the size of the BLOB returned by a geometry
function on the RHS of a MATCH operator in RTree.  The old code worked for
64-bit systems (by chance) but fails on 32-bit.

FossilOrigin-Name: 10cc44f5a63e6cadf6940bb7310a177ee7fa6ee2
2015-05-22 23:17:28 +00:00
drh fd29973f11 Update with fixes and enhancements from trunk.
FossilOrigin-Name: 9605d008f5c763137e9734d033fe4510ca316858
2015-05-22 18:40:48 +00:00
dan 6b76418e67 Have r-tree prefer to use the constraint "col MATCH ?" over "rowid = ?".
FossilOrigin-Name: b9fb95194d4f7c535f1d175fd2e18d69d76e8fc4
2015-05-22 16:08:42 +00:00
dan 0d45967b7f Increase test coverage of fts5_vocab.c.
FossilOrigin-Name: 065ab83a6ce36e16d3b95a61505aa3cff0bfea84
2015-05-22 07:44:44 +00:00
dan 21b7d2a9b8 Improve test coverage of fts5_unicode2.c.
FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e
2015-05-22 06:08:25 +00:00
drh 4f03f413bc Add the sqlite3_value_dup() and sqlite3_value_free() interfaces. Use
these interfaces to enhance R-Tree to add the
sqlite3_rtree_query_info.apSqlParam field.

FossilOrigin-Name: a7ee40c4fc62843ac5b96ba47ca14a66e8cd6961
2015-05-20 21:28:32 +00:00
dan 8c1f46de50 Improve test coverage of fts5_tokenize.c.
FossilOrigin-Name: 0e91a6a520f040b8902da6a1a4d9107dc66c0ea3
2015-05-20 09:27:51 +00:00
drh 7ac07bda73 Fix the MSVC makefile so that it works with OTA. Fix a harmless compiler
warning in FTS3.

FossilOrigin-Name: c7b16645307d6e46c4153693d6806269de64955f
2015-05-19 22:56:45 +00:00
drh 856d446efe Allow R-Tree geometry functions to take 8-byte BLOB arguments which are
passed directly through to the underlying callback, and which can be used
to pass pointers into the callback.

FossilOrigin-Name: b271ed56532a78323accc8a7cb348d55f95c350e
2015-05-19 22:20:48 +00:00
dan 116eaee4a0 Add tests for fts5 tokenizers.
FossilOrigin-Name: 4f90ba20e2be6ec5755fe894938ac97342d6fbf6
2015-05-19 19:37:09 +00:00
dan 815bd625c2 Remove the sqlite3ota_open_v2() API. Add a new parameter to sqlite3ota_open() instead.
FossilOrigin-Name: c74e0bc481ce995f83ca8384e05dfbe068a0ae85
2015-05-19 17:48:11 +00:00
dan a6fb464878 Allow OTA update state data to be stored in a database separate from the OTA update database.
FossilOrigin-Name: 5af8db56af457d60ea030d84666ca7fffb6821fe
2015-05-19 16:22:58 +00:00
dan b10210ea1b Fix a memory leak that could follow an OOM condition in fts5.
FossilOrigin-Name: de9f8ef6ebf036df5a558cd78fb4927da2d83ce8
2015-05-19 11:32:01 +00:00
dan 09963b27b8 Improve test coverage of fts5_storage.c.
FossilOrigin-Name: 4dfe2e5871704125338620705e6638f36baaacd7
2015-05-18 20:34:59 +00:00
dan 23390b1b2b Fix a bug in test script fts5fault4.test.
FossilOrigin-Name: a508059305641c2cb53197b0f6e93d7b723697e2
2015-05-18 18:24:41 +00:00
dan 5211e54f95 Add a test for an untested branch in fts5_expr.c.
FossilOrigin-Name: ce08206b5c5bf6a81decf20c99a524dd92c3715b
2015-05-18 18:03:06 +00:00
dan 27aac274b9 Improve test coverage of fts5_config.c.
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
2015-05-18 17:50:17 +00:00
dan e0569a4aae Further test coverage improvements for fts5.
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
2015-05-16 20:04:43 +00:00
dan 369e4129b9 Improve test coverage of fts5_index.c.
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
2015-05-15 18:13:14 +00:00
dan 415ae725cb Add test cases. And some fixes.
FossilOrigin-Name: adee788586197445672013d434e7ba47ce510b59
2015-05-15 12:18:39 +00:00
dan 8255079611 Merge latest trunk changes with this branch.
FossilOrigin-Name: 299ed55c909d327826ef47e204ef1e9afd54852b
2015-05-14 08:58:52 +00:00
dan 5d03b9d365 Merge latest trunk changes with this branch.
FossilOrigin-Name: b5f0e8c5b4bc018d672617ffd342d12b228548b9
2015-05-13 18:12:58 +00:00
dan 23d538885e Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token.
FossilOrigin-Name: aa34bf666c384cf32a8d8166ab6d9afbca26a256
2015-05-13 17:15:32 +00:00
dan 1c1cea8bae Merge latest trunk changes into this branch.
FossilOrigin-Name: 82e5a6e088c58815140ad36715ac11c96527cb25
2015-05-11 18:46:42 +00:00
dan 71ab324066 Allow the fts5vocab table to optionally provide data on a per-column basis.
FossilOrigin-Name: 3922276135a7825d0ede8d9c757e9cfe492f803a
2015-05-09 18:28:27 +00:00
dan 5e38f1c9bf Add the fts5vocab module, for direct access to the fts5 index.
FossilOrigin-Name: 6bf93e3b56e6705b7d12bab5024fc615f373b36c
2015-05-08 20:21:24 +00:00
dan 76724372ae Improve the error message returned by FTS5 if it encounters an unknown file format.
FossilOrigin-Name: f369caec145f311bb136cf7af144e2695badcb9b
2015-05-08 09:21:05 +00:00
mistachkin 6a12854024 Remove a couple stray test breakpoint calls.
FossilOrigin-Name: 2860cebeeaebd346de60c762aa3e51dbab008578
2015-05-08 00:58:39 +00:00
dan 4591334dd4 Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.
FossilOrigin-Name: a684b5e2d9d52cf4700e7e5f9dd547a2ba54e8e9
2015-05-07 19:29:46 +00:00
dan 739b903e6d Remove some dead code from fts3_snippet.c.
FossilOrigin-Name: 46b2d3cef5c22a9e6bd0a4f8411f17b7ec72bd18
2015-05-06 17:51:59 +00:00
dan e60aedc564 More optimization for the 'y' and 'b' matchinfo directives.
FossilOrigin-Name: 8c5b9fedfcee3ac22a222819dceb981ad94a9903
2015-05-06 17:41:19 +00:00
dan 753c1f56db Further optimizations for the 'y' and 'b' matchinfo operators.
FossilOrigin-Name: fbd038bb57322e1ed2e1ee52f3d134594b6bfcc0
2015-05-06 08:43:26 +00:00
dan 99eaf39784 Add the fts3 matchinfo 'b' flag.
FossilOrigin-Name: b9b77972d88171e4239b8194f308eb5d60b5d172
2015-05-05 20:39:53 +00:00
dan b2bd727e52 Optimizations for the matchinfo() function, particularly the 'y' flag.
FossilOrigin-Name: dddd7e182943a1d3a9d32830e819a63f1a228d6d
2015-05-05 19:37:07 +00:00
dan 7c479d51e5 Reorganize some of the fts5 expression parsing code. Improve test coverage of the same.
FossilOrigin-Name: c4456dc5f5f8f45f04e3bbae53b6bcc209fc27d5
2015-05-02 20:35:24 +00:00
dan 28f98455f4 Add the experimental matchinfo 'y' flag to fts3/4.
FossilOrigin-Name: 92941609af74044b3078e020324a37b04a0638b0
2015-05-02 09:44:15 +00:00
dan 0501b2492a Further improvements to test coverage of fts5 code.
FossilOrigin-Name: d4331943dff259380c4025bb740d8aba6972d351
2015-05-01 20:38:57 +00:00
drh e9c5f97683 Fix an assert in RTREE that would fire if the rtree table is misdeclared.
FossilOrigin-Name: 9a45409cc4078f2b6e68aa777f6ab86a14309833
2015-05-01 18:00:37 +00:00
drh abf582509d Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3,
and to ensure that error messages are not overwritten and thus leaked.

FossilOrigin-Name: 605347e087ec5eb817f3c94f4616abe26c7ab66b
2015-05-01 14:07:30 +00:00
dan 557085a5a1 Improve test coverage of fts5.c.
FossilOrigin-Name: add4f4681c648dcbecaa68d08f7b2f4e6d63003c
2015-05-01 12:14:23 +00:00
dan 7b2ec1ae41 Improve fts5 tests.
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
2015-04-29 20:54:08 +00:00
dan 90dd70226f Fix an fts5 bug in handling writes while there are active cursors.
FossilOrigin-Name: 07f70955392697556ca2951c9b6c3a5204cd5ec0
2015-04-28 20:24:50 +00:00
dan cdbd150170 Improve coverage of fts5 tests.
FossilOrigin-Name: 8e8136f2dc08082c2984462719d9cba0f212c92a
2015-04-28 18:35:28 +00:00
dan a3bdec7ee4 Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement.
FossilOrigin-Name: e38e2bb637844dae8ae5d5f3e23d8369e1b91e45
2015-04-27 16:21:49 +00:00
dan c1cea8f731 Further tests for fts5.
FossilOrigin-Name: ffeb3ef3cfec3681b72bb28cfa612aa15e07887d
2015-04-27 11:31:56 +00:00
dan 26c84373c4 Improve coverage of fts5_index.c slightly.
FossilOrigin-Name: e5aaa01306597ffd2475dcb83ae889393f68d315
2015-04-25 20:29:46 +00:00
dan b33e64c83e Add tests for fts5.
FossilOrigin-Name: e748651c940eae2389fe826cf5c25f1166a5e611
2015-04-25 18:56:48 +00:00
dan ef8b74324d Merge latest trunk changes with this branch.
FossilOrigin-Name: 1c78d8920fb59da3cb97dd2eb09b3e08dfd14259
2015-04-24 20:18:21 +00:00
dan df5bd1fed2 Add the "unindexed" column option to fts5.
FossilOrigin-Name: 86309961344f4076ddcf55d730d3600ec3b6e45c
2015-04-24 19:41:43 +00:00
dan def90aae18 Add extra tests for corrupt database handling in fts5.
FossilOrigin-Name: 41449f7a0b5da6332eef48386c91ef63382c4783
2015-04-24 15:56:09 +00:00
dan 6758959ec3 Fix a problem in the "ota" utility program causing it to crash if SQLite failed to open the target database.
FossilOrigin-Name: a1a2b4a2eff4f641fa80e8ffa805c97afb73be95
2015-04-23 19:38:20 +00:00
dan e11afed562 Merge latest trunk changes into this branch.
FossilOrigin-Name: 1506e99eca19f72fb59b55ec9e2534505dfa7a6e
2015-04-23 19:32:19 +00:00
dan 55d46f01c5 Fix a performance problem in calls to sqlite3ota_close() made after the OTA update has been completely applied and checkpointed.
FossilOrigin-Name: fa62093b2531424846ea500c4155170d1bc7110e
2015-04-23 19:18:42 +00:00
dan 1cdd523a50 Have OTA maintain a small LRU cache of UPDATE statements. This reduces the amount of time it spends compiling UPDATE if a single data_xxx table contains many different ota_control strings.
FossilOrigin-Name: baee3556ea10d96f1623cf4dce112fa1a1070820
2015-04-23 18:14:21 +00:00
dan 38f2024062 Fix a problem causing the fts3 integrity-check to fail if run inside a transaction.
FossilOrigin-Name: 3b925189a75eae875da256b6e54999cad3cef1eb
2015-04-23 11:52:04 +00:00
dan 206480992d Add extra OOM tests for fts5.
FossilOrigin-Name: 2dd59b5762c944b3bdd270e95c6739cd4f530bfa
2015-04-22 20:58:24 +00:00
dan c51f5b3fd9 Fix problems with doclist-indexes involving very large rowids.
FossilOrigin-Name: 1e5994097e4c740c5173ea9718c3935728fdb86f
2015-04-22 20:14:46 +00:00
dan 39b7ff7987 Add comments related to database locking to sqlite3ota.h.
FossilOrigin-Name: 77242965e77446313a8f0a65a39fccb67fe4cabf
2015-04-22 11:34:38 +00:00
dan 062c267dff Update this branch with latest trunk changes.
FossilOrigin-Name: 9797482ded7de985e3b20aedec5e4d81f55065c8
2015-04-22 09:40:35 +00:00
dan 5bb97c2b80 Fix a problem in fts5fault1.test.
FossilOrigin-Name: a21d60cb2ac6463c012d82d1970d90da5da2a14a
2015-04-21 20:13:38 +00:00
dan d7889b26ac Fix an fts5 problem with large deletes.
FossilOrigin-Name: e50e8031d6f804ebe50e0eec9a8b6e7f0152ecc3
2015-04-21 19:07:39 +00:00
dan 903dd7d5d1 Add a comment to sqlite3ota.h. No code changes.
FossilOrigin-Name: 352fa09efd8240567b8f9487944e8c3ade4e664a
2015-04-21 15:35:30 +00:00
dan 3c9a073723 Fix an fts4 problem to do with the handling of phrases that contain zero tokens.
FossilOrigin-Name: caba5be0b2bdaf4eb4af9af267c0c22dfbd3dba5
2015-04-21 12:06:53 +00:00
dan 80d3ea080a Fix some fts5 problems with very large position lists.
FossilOrigin-Name: 2ea8f9cbe67dac60c1a0a661c95a03ecfa9a0b9a
2015-04-20 18:48:57 +00:00
mistachkin 8e18922f12 Fix harmless compiler warnings seen with MSVC 2015.
FossilOrigin-Name: d05becd873a03a366843a34e7f4c732dd8f88c86
2015-04-19 21:43:16 +00:00
dan 1586699cfc Fix a problem affecting some fts3 UPDATE and DELETE statements on order=DESC tables.
FossilOrigin-Name: 284c1623f81704ef80edb9324954525cb2b72172
2015-04-17 20:51:55 +00:00
dan 1c29c27bb0 Ensure that filenames passed to the VFS xOpen() function are terminated by 2 '\0' bytes.
FossilOrigin-Name: 66eff473c3634edb42f6ad6955acfba3e3a10def
2015-04-17 17:00:52 +00:00
dan 43030868e6 Fix a memory leak in sqlite3ota.c that can follow an OOM error.
FossilOrigin-Name: c3dc15e71782fc70e92b71ad9eec0cf8ffef611f
2015-04-17 16:29:11 +00:00
dan ec16d98489 Have OTA always specify SQLITE_CONFIG_URI when opening databases. Fix a test issue causing otacrash.test to fail.
FossilOrigin-Name: 0d0e5ec064eaecb200b9b601b7a54a1700cd176e
2015-04-17 08:36:05 +00:00
drh a4bb7764fe Fix a boundry-value condition in the phrase poslist extractor of FTS3/4.
FossilOrigin-Name: 55d10baf0bffdb1a34bf5627ed8f25e4a4efd942
2015-04-16 20:10:17 +00:00
dan 5c06dd831f Be sure to release any xShmLock locks held when closing an OTA handle.
FossilOrigin-Name: d0fba72a47f73082ade40a732aab114bc8f6d83e
2015-04-16 18:49:53 +00:00
dan 28b22a0159 Fix a mismatched printf() argument and format specifier. Add ota files to Makefile.in.
FossilOrigin-Name: 5db810a88dac5cbf1f6b321345d01cc0d9d05d31
2015-04-16 14:33:09 +00:00
dan 4141a81b30 Avoid attempting to rename the *-oal file while it is still open.
FossilOrigin-Name: 658c3b0c80b9fe7eaac5982573651dc649a4c53f
2015-04-16 10:55:58 +00:00
dan 687c5124e0 Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances.
FossilOrigin-Name: 50fae1f0006c0e946b5214e73eedf2687a0016f9
2015-04-15 18:49:20 +00:00
dan dbca0ab9b0 Fix a problem preventing doclist indexes from being loaded.
FossilOrigin-Name: b29109a083e5cd442cdd19f29d7be45b09c4f661
2015-04-15 16:01:42 +00:00
drh fdb031891c Merge all recent trunk enhancements and fixes into the ota-update branch.
FossilOrigin-Name: 9bd3e4453d4ad416f7e3f08f0bd283d34f1c319c
2015-04-15 14:26:04 +00:00
dan 0d01ec8de8 Remove an incorrect assert() statement from sqlite3Fts3Dequote().
FossilOrigin-Name: bd06eeb8d06237dc2d54d8a03e8bf525cb811c9e
2015-04-15 09:16:39 +00:00
dan 2deb12be2b Fix a problem causing an assert() to fail if a snippet containing 0 tokens was requested from fts3.
FossilOrigin-Name: eafd0a1e3f25f38d551603f177ff4634cf79de77
2015-04-15 08:20:50 +00:00
dan f1e58b7e8d Begin changing fts5 to use a delete flag so that delete markers may be annihilated more quickly.
FossilOrigin-Name: 9341c070bb6140dbf559680952909674aa83fa55
2015-04-14 20:15:41 +00:00
dan 929b2cfc2e Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index.
FossilOrigin-Name: bdb8e82ab683f2438cde9f0b63e497dbf0141dcf
2015-04-11 18:25:04 +00:00
dan 56e2497aaa Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.
FossilOrigin-Name: 49c1e74522a26e5dbe6f8305bc96487279b80dfb
2015-04-11 16:23:31 +00:00
dan 75395ccd17 Do not allow virtual table constructors to be called recursively.
FossilOrigin-Name: 0a72726da21581ab16cb3e964bd825b8f2e931e4
2015-04-10 07:55:07 +00:00
drh 57d0d9f33c Merge all changes for version 3.8.9 into the ota-update branch.
FossilOrigin-Name: ec9d907a57fcea72c8a64e521e8a1b5777d67568
2015-04-08 14:01:07 +00:00
dan 4b0000c9a8 Fix a problem with fts3 prefix terms within phrase queries on "order=DESC" tables with a mix of negative and positive rowids.
FossilOrigin-Name: 0cdf502885ea7e5805d7ba3719f055f5d48fc78d
2015-04-06 09:05:29 +00:00
drh f3e8748683 Fix a harmless compiler warning in FTS3.
FossilOrigin-Name: 1688f60c08cc9270948402d98a2333382aa32681
2015-03-26 14:29:51 +00:00
dan 6da7a0a93d Merge the latest trunk changes into this branch.
FossilOrigin-Name: 9d9b6c883b4f7d69c615cedfb59a2385aac47b74
2015-03-24 18:21:41 +00:00
dan f37120a416 When the OTA module updates or deletes a row, save only those fields that are part of an index or primary key to the ota update database.
FossilOrigin-Name: 6326fd3249bee267da0172f8afd1e7b3f71521b9
2015-03-24 18:03:29 +00:00
dan 5e4eeb3c00 Fix a broken assert() in the ota module.
FossilOrigin-Name: 858de8a5e7925bd96d7b9c0f00bc16358cf3b295
2015-03-23 17:10:51 +00:00
drh 3858cb44b6 Add a missing "int" on a constant declaration.
FossilOrigin-Name: 235157de5113ac4c750e36a498e2a1f1cf461751
2015-03-21 20:50:58 +00:00
dan e10d87f65a Avoid a segfault if NULL is passed as the first argument to SQL scalar function fts3_tokenizer().
FossilOrigin-Name: 6d0989695b486275824c14d5f88357267c1e8104
2015-03-21 19:35:09 +00:00
dan 71138b0f0f Merge trunk changes with this branch.
FossilOrigin-Name: 142743918fd5a6d79fa10c44398360c3684a255a
2015-03-21 15:45:24 +00:00
dan 8885708c3c When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.
FossilOrigin-Name: ce972f6aab90f6929d018696f1ab3c2649eca802
2015-03-21 15:37:19 +00:00
dan 998aaa03ea Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test.
FossilOrigin-Name: e3e234649616f20610abce9ae9da1c572d3a4377
2015-03-21 12:22:51 +00:00
mistachkin 44723ce096 Improvements to the MSVC build. Fix harmless compiler warnings. Enable use of 'stdcall'.
FossilOrigin-Name: 737630b87314283b2c38790ace9d25ec05f81f4a
2015-03-21 02:22:37 +00:00
dan b7134d0863 Fix an FTS3/4 problem with handling empty tokenizer declarations (e.g. "CREATE VIRTUAL TABLE t(tokenize=);").
FossilOrigin-Name: 26d2def8a53094356008861636d66f9ae8f2448a
2015-03-19 16:25:42 +00:00
dan e0417626ea Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer.
FossilOrigin-Name: ad4b19d2ac0889a23fe3b0fd844286efc10cdd82
2015-03-19 15:52:07 +00:00
dan c3b328da6e Allow whereShortCut() to use the PRIMARY KEY index of a WITHOUT ROWID table to optimize a vector of "IS" operators in a WHERE clause.
FossilOrigin-Name: 52e73eeca063bb30092ce600068bf487641399a0
2015-03-18 20:03:27 +00:00
dan 9340de52e2 Clarify that OTA is unable to update or delete rows with NULL values in primary key fields.
FossilOrigin-Name: 2e7c1e0a0d128d9bac119692b2505b5ed0abe87a
2015-03-18 19:04:40 +00:00
dan f5fab92d82 Add an optimization to the fts5 unicode tokenizer code.
FossilOrigin-Name: f5db489250029678fce845dfb2b1109fde46bea5
2015-03-11 14:51:39 +00:00
dan c9ca6fb85d Avoid redundant string comparisons while merging fts5 segment b-trees.
FossilOrigin-Name: 5c46820d9b4aae791a8704b69145bd81f1e6780d
2015-03-10 19:24:30 +00:00
drh a9106ca83d Merge recent trunk enhancements into the ota-update branch.
FossilOrigin-Name: 5489cb68921f62f10d832adbc4d19ea8c6c5da50
2015-03-09 13:24:46 +00:00
dan 92faed663a Fix some compiler warnings caused by signed/unsigned pointer conversions.
FossilOrigin-Name: cccee7b5b1e84523f1c549d3052fd170e32bde80
2015-03-07 15:46:41 +00:00
dan 6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
dan 54fc21438e Fix some problems with OTA and empty target databases, or target databases with the wrong set of tables. Also add SQLITE_FCNTL_VFSNAME support to the OTA VFS.
FossilOrigin-Name: 46119e8d8e391d8dea844352521b58415f7365b1
2015-03-05 16:21:20 +00:00
dan 2bd7ace54e Add comments to sqlite3ota.h to make it clear that passing NULL in place of a parent VFS name to sqlite3ota_create_vfs() causes the new VFS to use the system default as its parent.
FossilOrigin-Name: 158c1a48818a9abc001b9ea547167c2624a7bad3
2015-03-05 14:07:25 +00:00
dan 67d4b20c46 Fix a typo in sqlite3ota.h.
FossilOrigin-Name: 04087dec4c3db5f322eca289585525b7267ed4f8
2015-03-05 13:43:21 +00:00
mistachkin 0b7ff5c11b Fix harmless compiler warning.
FossilOrigin-Name: 580dae4615353d73c7d096107571bd60fec5b0f2
2015-03-04 20:18:55 +00:00
dan 47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan f400282bdf Further minor optimizations to flushing fts5 data to disk.
FossilOrigin-Name: a07dcca9ef3821a6719ef9dbbc8ed861fa005035
2015-02-27 09:41:10 +00:00
dan 5db7e2ca83 Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.
FossilOrigin-Name: bb104b3646c6f07ed002be7360b08433ee7980d4
2015-02-27 07:23:26 +00:00
mistachkin 38e40ee2e5 Fix potential memory leaks in the misc 'compress' extension.
FossilOrigin-Name: 3bc34fd427d9d7819cd9740237b1f5d4180341fa
2015-02-26 21:04:44 +00:00
dan dfdc4b4613 Optimize copying data from fts5 in-memory hash tables to top level segments.
FossilOrigin-Name: 8e3ca6323a2beab5f04250e24ae15b159d2aa0ac
2015-02-26 20:49:09 +00:00
dan 4a7e11c629 Fix an fts5 bug in large incremental merges.
FossilOrigin-Name: 208e3cb6b6dc8c7d824b64dec2034004c9fcbba5
2015-02-26 14:54:03 +00:00
dan 6d8f42ec81 Instead of the 4-byte fields, use regular varints for the poslist-size field in fts5_hash.c.
FossilOrigin-Name: 7eb022d7e5fdb180af823c82c47c938e4a7a355f
2015-02-25 19:24:37 +00:00
dan 5b828f8c33 Fix building with ENABLE_OTA and OMIT_LOAD_EXTENSION.
FossilOrigin-Name: d6d21ff55228d3ffe6cdbb5755645b147806a1da
2015-02-23 20:09:50 +00:00
dan 04f121c187 Change SQLITE_FCNTL_ZIPVFS_PAGER to SQLITE_FCNTL_ZIPVFS.
FossilOrigin-Name: f7865b942834dd2f6b865336e08ba1adbf20612a
2015-02-23 15:41:48 +00:00
dan 5ebc83b1d2 Update the sqlite3ota_db() API to account for the fact that each OTA handle now uses two SQLite database handles.
FossilOrigin-Name: ef08ecceb7e237a01af6cc3141dccee09ffe9ae3
2015-02-23 15:02:13 +00:00
dan 882b8e0974 Improve tests for resuming ota updates following power failures. Fix a problem revealed by the same.
FossilOrigin-Name: 1cb675e5392f179516d8e7a52760922a6c7df4d0
2015-02-23 12:22:55 +00:00
dan ee9fa4af15 Fix some problems with resuming ota updates if saving the state is interrupted by a power failure or system crash.
FossilOrigin-Name: 6d5ed70d0dbabe9c2ab2f2dba47747d17d937781
2015-02-21 20:08:25 +00:00
dan 92f55ceb70 Add some missing comments and fix other minor code issues in sqlite3ota.c.
FossilOrigin-Name: 718fd8b673d6557dd0eaad03e6a3332b5490afbf
2015-02-20 14:36:16 +00:00
dan 942b0a68ea Ensure the mutex used to protect the linked list of all main database files opened by a single ota vfs is allocated.
FossilOrigin-Name: 9c8682d6650a94e11f9bec5baff69ed9668874fa
2015-02-19 19:59:35 +00:00
dan 138bf3bc7f Update various documentation comments in sqlite3ota.c and sqlite3ota.h.
FossilOrigin-Name: 60e0a46b82dd9c704e8aa977d1ccdd73d388422f
2015-02-19 18:06:40 +00:00
dan 20ab217f93 Add tests for a couple of previously untested branches in the ota code.
FossilOrigin-Name: a3c1bc5d5e3f4b197f48cbbc240608e94bfc2b45
2015-02-19 13:36:02 +00:00
dan 7551c5c3e6 Add new file ota12.test, containing tests for applying ota updates to live databases with other active reader/writer clients.
FossilOrigin-Name: 0864d127fe42fc0db7ab30a3ebf74c0114095648
2015-02-18 20:17:14 +00:00
dan e3e031ce85 Add ota tests to increase code coverage. Fix some minor issues in error handling within the ota code.
FossilOrigin-Name: 2b10c5d2b8b8b535d3dec0c68a777db16268e1e5
2015-02-18 20:16:15 +00:00
dan d092df51de Fix a problem with OTA updates in the presence of database readers.
FossilOrigin-Name: 144bb29ffcbfe96dc10c0224113e73a80e89314b
2015-02-18 17:40:05 +00:00
dan eadc772beb Improve test coverage of ota code a bit.
FossilOrigin-Name: a438fa6c9ad2fb1d78ac747172d07455d6381387
2015-02-17 20:49:42 +00:00
dan f5fe23909d Add extra tests and fixes for ota.
FossilOrigin-Name: e0b7151962fedbcac975f2216fd6b33b995a8945
2015-02-16 21:13:19 +00:00
dan 5902352fca Add further tests and fixes for ota.
FossilOrigin-Name: 62dc1fffc38cb157c15105098749b6dd0198eb84
2015-02-16 11:48:34 +00:00
dan c76c64f931 Move tcl test code from sqlite3ota.c to new file ext/ota/test_ota.c.
FossilOrigin-Name: f20779a6e890ba73bfaa904cefcf3a029b01fed4
2015-02-16 06:27:37 +00:00
dan 52800df9fc Change the way the "incremental checkpoint" function of OTA works in order to reduce the effect on the SQLite core code.
FossilOrigin-Name: b64a11a754dc56f3406d3b703531ebe9e4af4908
2015-02-14 18:58:22 +00:00
dan 01325e86e6 Merge the ota-update-no-pager_ota_mode branch into this one.
FossilOrigin-Name: 71887cd9b38def398d48eaf0ec34eeac3c7c5177
2015-02-11 17:05:17 +00:00
dan a3cf4f7ccb Merge latest trunk changes with this branch.
FossilOrigin-Name: 0b63e8dcbaec5043e353734e684c2a46552a3409
2015-02-11 16:54:48 +00:00
dan 2e24971f2a Ensure that an error is reported if an attempt is made to update a wal mode database via ota.
FossilOrigin-Name: 6fc5d4d26a603b3906f02ceea0f507780d0c35eb
2015-02-11 16:25:27 +00:00
dan cd73244144 Further tweaks to work with zipvfs.
FossilOrigin-Name: 0f152416be792457c52417aeb531ac860d12a5bd
2015-02-10 20:00:38 +00:00
dan 07427d0350 Add documentation and test cases for sqlite3ota_create_vfs(). Also code to detect errors in zipvfs/ota setup.
FossilOrigin-Name: e729668168f00325459bc2e9b515aa95e57f2754
2015-02-10 17:08:17 +00:00
dan faa8c4fdcb Add the sqlite3ota_create_vfs() and sqlite3ota_destroy_vfs() functions.
FossilOrigin-Name: 96443ecb6909141aa621a16e628455857d036482
2015-02-09 20:07:35 +00:00
dan de1f6a270b Add comments to explain the role of the ota vfs.
FossilOrigin-Name: 7bb633639d1e41067903a49653f09a823054e213
2015-02-07 20:20:32 +00:00
dan dd0ad3785b Remove "PRAGMA pager_ota_mode".
FossilOrigin-Name: 8ac58e46782bd6b81c06fdf1cb5b316b8a4e1ddf
2015-02-07 19:17:36 +00:00
drh 2f8109cef1 Revise the way that the index structure for a WITHOUT ROWID table is
discovered.

FossilOrigin-Name: 7f10a0eaf1fedfa020cbd7019ec9342ffdc3b9b0
2015-02-06 00:31:45 +00:00
dan 8800b53bc5 Change a comment in sqlite3ota.h to make it clear that it is not possible to insert a NULL value into an INTEGER PRIMARY KEY column using ota.
FossilOrigin-Name: a5e86bea4ad51bbb988a2a2215961706894f4a02
2015-02-05 17:46:19 +00:00
dan a8ccf3ed74 Prevent ota updates from violating NOT NULL constraints. Add a comment to the "limitations" section of sqlite3ota.h saying that CHECK constraints are not enforced.
FossilOrigin-Name: 74e073dd604142212f3d3e1931065d124daabd80
2015-02-05 17:36:30 +00:00
drh d4ef7c50bb Figure out the primary-key type of a table using queries of sqlite_master
and the table_info and index_list pragmas, obviating the need for
SQLITE_TESTCTRL_TBLTYPE.

FossilOrigin-Name: 50ecdfc443b51e3569c6add2fba5132f959c61cb
2015-02-05 01:49:31 +00:00
dan 06c9722564 Fix the error messages returned by ota if an update violates a unique constraint.
FossilOrigin-Name: c4845a3b591723ff60b7b1da198f44908c12bdfc
2015-02-04 19:20:42 +00:00
dan 738a3b2fcb Avoid a linear scan of the entire table when ota updates or deletes a row from a table with an external primary key index.
FossilOrigin-Name: 1db198ccca1e5c5a922cefe3daeff8d2e5d3a7f7
2015-02-04 16:32:47 +00:00
dan 1e35850229 Fix a memory leak that could follow an OOM error in ota.
FossilOrigin-Name: 0d5415f26427d0af89663c5a3e944e488143b648
2015-02-04 11:08:47 +00:00
dan e2863b6074 Fix some problems surrounding WITHOUT ROWID tables with DESC primary key indexes
.

FossilOrigin-Name: a21fefb79a161e6bb057ae4f6ba554f659706ac1
2015-02-03 18:43:42 +00:00
dan 75fda9b395 Remove "PRAGMA ota_mode".
FossilOrigin-Name: 1c111447a07687c30ed4ad5a6c27a169c85b7ea6
2015-02-03 15:56:08 +00:00
dan 57fec54b53 Fix some problems with building fts5 and fts3 together using the amalgamation.
FossilOrigin-Name: fb10bbb9f9c4481e6043d323a3018a4ec68eb0ff
2015-02-02 11:32:20 +00:00
dan 32348f7bd2 Merge latest trunk changes with this branch.
FossilOrigin-Name: 76212f2c9a3c3ff0e238d6dad776938c6af674e6
2015-02-02 09:40:36 +00:00
dan 3b660d7da3 Have ota use imposter tables to write to indexes instead of the sqlite3_index_writer() interface. The error handling in this version is broken in a few small ways.
FossilOrigin-Name: cdaeab467f6aa3217be161377a9b78a4eec37093
2015-01-31 20:42:04 +00:00
dan 5082140311 Minor optimizations to fts5 writes.
FossilOrigin-Name: 1fffe51fa92f1784365140d5b163ab6c690981ae
2015-01-31 15:23:44 +00:00
drh 3102aa01fd Merge all recent trunk changes, and especially the SQLITE_TESTCTRL_INITMODE
enhancement.

FossilOrigin-Name: 36436dde74ce2536a9a430b4458216499ad113bf
2015-01-30 16:36:17 +00:00
dan 719b0f66e5 Fix some problems with transactions that both read and write an fts5 table.
FossilOrigin-Name: 0e225b15357765f132c3364b222f9931a608a5b2
2015-01-29 20:59:34 +00:00
dan 88392bf3ce Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query.
FossilOrigin-Name: 85dc12625d300fe48f3c096f54ebcb8b6ef4e30a
2015-01-29 11:52:22 +00:00
drh 120b3a4f61 Merge in all changes from trunk.
FossilOrigin-Name: 17c69be80542c5f84e21d60df3edc49422b087d9
2015-01-28 12:00:40 +00:00
mistachkin e4a0d79b8a Fix harmless compiler warnings.
FossilOrigin-Name: e7d2ec048c88237c124fbe598f8f7e950d43d90f
2015-01-27 21:24:33 +00:00
dan 900f1922ac Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.
FossilOrigin-Name: f704bc059e06b01f1d68fa7dad89e33eace6c389
2015-01-27 20:41:00 +00:00
dan 6f0138e89e Fix a bug in the fts3 snippet() function causing it to omit leading separator characters from snippets that begin with the first token in a column.
FossilOrigin-Name: adc9283dd9bc3a6463f8c4fe23dd58a3712c349d
2015-01-27 19:01:26 +00:00
dan 18f6ff9eb7 Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function.
FossilOrigin-Name: 245e8730451fbdc1c729beff7295c452df604009
2015-01-27 18:43:02 +00:00
dan 8ac3025419 Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.
FossilOrigin-Name: 5206ca6005bfa9dfc7346d4b89430c9748d32c10
2015-01-24 19:57:03 +00:00
dan 641cb4360a Fix compression of keys stored on internal segment b-tree nodes by fts5.
FossilOrigin-Name: 51444f67c0cc58a3023eb1cd78e7cf889da6c80f
2015-01-23 17:43:21 +00:00
dan 626d9e3062 Remove some redundant code from fts5.
FossilOrigin-Name: 939b7a5de25e064bdf08e03864c35ab718da6f6f
2015-01-23 06:50:33 +00:00
dan 0b520cc5f9 Add further tests and fixes for fts5.
FossilOrigin-Name: 5b295897153e9b26cd0d2e7ea112a4d461d0a665
2015-01-22 19:13:08 +00:00
dan 4c2871bead Further tests and fixes for fts5.
FossilOrigin-Name: c020a291ed293a66d21c5885e50a7fee04aa6366
2015-01-21 20:30:14 +00:00
dan dd8a501019 Fix an fts5 issue with loading doclist-indexes for a term that is the last thing on its leaf page.
FossilOrigin-Name: e0d614425f7f5ffe266fdc03642931b1ec19ad25
2015-01-21 18:23:25 +00:00
dan b461380333 Add further tests for fts5 backend.
FossilOrigin-Name: 09dabb3b9e140eec6cfda83bcb86b6b9f5cf54b6
2015-01-21 16:10:59 +00:00
dan 997d798d92 Merge latest trunk changes with this branch.
FossilOrigin-Name: b3348b1e07e168b156636a29fc8c6d6afb3129c2
2015-01-21 06:36:07 +00:00
dan fd0b436e10 Add extra fault injection tests to fts5.
FossilOrigin-Name: f45a0dc0a8911c8aac5a1028ac4f543a709656e7
2015-01-20 20:34:17 +00:00
dan aa4d380a42 Handle the case where a tokenizer determines that there are zero tokens in an fts5 query term.
FossilOrigin-Name: 75f3d17f864072dfa2caee182b86cc4b9972d691
2015-01-19 11:15:36 +00:00
drh 65545b5993 Eliminate all use of sprintf(), strcpy() and strcat() from test logic because
OpenBSD hates those functions.

FossilOrigin-Name: 10321910990195878c0af1e94b34ae0cdc0cb31b
2015-01-19 00:35:53 +00:00
dan d8736bc3b8 Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration.
FossilOrigin-Name: f30afd209aa4ce42766b1493750c4f5b5f1e9502
2015-01-17 20:01:52 +00:00
dan 2656167f6e Improve the performance of the fts5 porter tokenizer implementation.
FossilOrigin-Name: 96ea600440de05ee663e71c3f0d0de2c64108bf9
2015-01-17 17:48:10 +00:00
drh 068a251d99 Simplify some code in rtree, to avoid confusing the optimizer in GCC on
some macs:
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00).
Prior to these changes, compiling with -O3 would cause incorrect code to
be generated.  The change to the nodeGetCell() routine is key.  The other
changes are merely cosmetic details discovered while bug hunting.

FossilOrigin-Name: 882181ff9dd75f32db266db6e476671021fc567b
2015-01-13 21:26:17 +00:00
dan 851ca6e715 Fix prefix indexes so that they work in characters, not bytes.
FossilOrigin-Name: af8d43a4a08528bbae25ee38fe25de8a86f8a21c
2015-01-13 17:25:08 +00:00
mistachkin 7bb22ac7ff Silence some harmless uninitialized local variable warnings.
FossilOrigin-Name: 44375a34e59d1b0f0d765a99dcc7e6f30a4be7f7
2015-01-12 19:59:12 +00:00
mistachkin 7bb6e8e169 Get things compiling cleanly with MSVC and W4.
FossilOrigin-Name: c8725fa5fa361959b0f0a3fb36f204905d6f0ae9
2015-01-12 18:52:41 +00:00
mistachkin 1a51ce78ba Fix some harmless compiler warnings.
FossilOrigin-Name: 2b8eecbfe7935d3d9826fbcd473dd3a49138ad11
2015-01-12 18:38:02 +00:00
dan 73f7d6ed75 Optimize the unicode61 tokenizer so that it handles ascii text faster. Make it the default tokenizer. Change the name of the simple tokenizer to "ascii".
FossilOrigin-Name: f22dbccad9499624880ddd48df1b07fb42b1ad66
2015-01-12 17:58:04 +00:00
dan 27277c4e3c Fix some documentation issues in fts5.
FossilOrigin-Name: 512e1bdb4093b59d1494dfc63391476eadd52aea
2015-01-10 20:34:27 +00:00
dan d6b1c880d2 Add the fts5 'optimize' command.
FossilOrigin-Name: e749be563d8e738af113bd301770e2f22763ab77
2015-01-07 19:33:11 +00:00
dan 96a32bb51c Add the 'rebuild' and 'delete-all' commands.
FossilOrigin-Name: 0cb2fed525778d96237b5b0943047665e1f636d1
2015-01-07 17:11:11 +00:00
dan aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
2015-01-06 19:08:26 +00:00
dan 2a28e507f7 Further fixes and test cases related to external content tables.
FossilOrigin-Name: ce6a899baff7265a60c880098a9a57ea352b5415
2015-01-06 14:38:34 +00:00
dan ded4f41d1a Tests and fixes for fts5 external content tables.
FossilOrigin-Name: 047aaf830d1e72f0fdad3832a0b617e769d66468
2015-01-05 20:41:39 +00:00
dan 0fbc269fef Add support for external content tables to fts5.
FossilOrigin-Name: 17ef5b59f789e9fa35c4f053246d819987fd06f8
2015-01-03 20:44:58 +00:00
dan ade921c3ad Allow the rank column to be remapped on a per-query basis by including a term similar to "rank match 'bm25(10,2)'" in a where clause.
FossilOrigin-Name: 1cd15a1759004d5d321056905dbb6acff20dc7d9
2015-01-02 14:55:22 +00:00
drh 4081d5da3b Add logic to releasetest.tcl that detects -fsanitize=undefined errors.
Fix a few false-positivies that come up when running the sanitize=undefined
test.

FossilOrigin-Name: 2835e79a0afec6e4d449ac9340afec068c2d4c11
2015-01-01 23:02:01 +00:00
dan 37db72f1f7 Merge latest trunk changes with this branch.
FossilOrigin-Name: 4b3651677e7132c4c45605bc1f216fc08ef31198
2015-01-01 18:03:49 +00:00
drh f0a88279bd Fix a harmless compiler warning in rtree.
FossilOrigin-Name: 30891c6b8ebe9dfc939f9695bb45a159fbaaf262
2015-01-01 16:47:43 +00:00
dan 6024772ba2 Add a version of the unicode61 tokenizer to fts5.
FossilOrigin-Name: d09f7800cf14f73ea86d037107ef80295b2c173a
2015-01-01 16:46:10 +00:00
dan e716aca24b Move all fts5 test files to new directory "ext/fts5/test".
FossilOrigin-Name: 7f148edb30103c5f4fee20cd08e38537f9615bf2
2014-12-29 15:59:36 +00:00
dan 5fa3acabf4 Fixes to built-in tokenizers.
FossilOrigin-Name: b33fe0dd89f3180c209fa1f9e75d0a7acab12b8e
2014-12-29 11:24:46 +00:00
dan 2a615fa627 Fix the fts5 bm25() function so that it matches the documentation.
FossilOrigin-Name: 1ac7a8d0af9a71ddf6a1421033dcb9fa67c6120c
2014-12-23 19:18:34 +00:00
dan 005e10e39a Fixes and simplifications for the snippet() and highlight() functions.
FossilOrigin-Name: ca5d44042aa7461dcc8b700b0763df4df9d4a891
2014-12-22 21:01:52 +00:00
dan 1616d55153 Remove the fts5_test() aux function. Test aux functions using the tcl interface instead.
FossilOrigin-Name: 67e3ffd950c5347d219a06b33ad51949cffa7d90
2014-12-19 20:53:51 +00:00
dan 2ae0f4c931 Fix a problem with prefix queries and the AND operator.
FossilOrigin-Name: 38b3c65e3ee95eb7afadb76e0110570fbbc41e1b
2014-12-18 20:01:15 +00:00
dan e4bec37900 Fix various problems in fts5 revealed by fault-injection tests.
FossilOrigin-Name: e358c3de5c916f2c851ab9324ceaae4e4e7a0fbd
2014-12-18 18:25:48 +00:00
dan 0a99e9fb01 Update this branch with latest trunk changes.
FossilOrigin-Name: 69a312ad3fe5b39bc394b9ce958cb63d734518c7
2014-12-08 07:50:31 +00:00
dan fdf3a8970d Update comments in sqlite3ota.h to remove the "must have PRIMARY KEY" restriction.
FossilOrigin-Name: 088a41eb8c18886a260cf53fa0cca3bd1958dc05
2014-12-08 07:28:26 +00:00
dan 61826cdb6c Extra tests for the ota_rowid column.
FossilOrigin-Name: 46069393b3141ab198f0fcc4f6c05229f06bf1e8
2014-12-08 07:22:34 +00:00
dan 7ca1d07aff Allow the ota extension to write to tables with no PRIMARY KEY declaration.
FossilOrigin-Name: ba59a7e2ba97244492cbca9247456df0f3f19248
2014-12-06 19:30:41 +00:00
drh 2c3abeb8c3 Fix compiler warnings.
FossilOrigin-Name: e9955c0e14d13ba1411f013acb4979958dae2516
2014-12-05 00:32:09 +00:00
drh 063970a15f Fix comment typos reported on the mailing list. No changes to code.
FossilOrigin-Name: 93a71c9f058b58d11187293a1e8d5324d0911880
2014-12-04 14:01:39 +00:00
dan 845d0ab323 Begin testing fts5 OOM and IO error handling.
FossilOrigin-Name: 2037dba62fdd995ad15b642abe499a790f5ffe5c
2014-12-03 17:27:35 +00:00
dan 615a9ae5db Add a configuration option to remap the "rank" column to an auxiliary fts5 function.
FossilOrigin-Name: b5f5971283b9b2f60c16f9675099855af95012cd
2014-12-02 20:18:11 +00:00
dan 89a89560d6 Add code to parse a rank() function specification. And a tcl interface to add auxiliary functions to fts5.
FossilOrigin-Name: 9c1697a2aa1f601e6eb11704abe63a73c8105447
2014-12-01 20:05:00 +00:00
dan ca7fad3d2c Add a cookie mechanism to ensure that the %_config table is re-read as required.
FossilOrigin-Name: bb4a37b53de60da9ec8b9317eec14afa99690828
2014-11-28 20:01:13 +00:00
dan cb11e73fc2 Add a %_config table to fts5.
FossilOrigin-Name: 83491c56661ca78f96020ba68184bb3fb19e674f
2014-11-27 20:03:45 +00:00
dan ca3d648f9c Update ota so that the hidden columns of virtual tables may be written.
FossilOrigin-Name: ccee999649d0fa1d48e53847542f4cbe05e3d694
2014-11-27 18:09:46 +00:00
dan cb62aae034 Add the auxiliary highlight() function to fts5.
FossilOrigin-Name: 059092379f981eb919b500ce447006f9e645fc5a
2014-11-24 16:24:33 +00:00
dan 7bf9ec1c26 Add SQLITE_ENABLE_OTA pre-processor directives so that this branch may be compiled with or without OTA.
FossilOrigin-Name: 600cefdd4d29c1de4d107fa7ddeb76a18edce4f5
2014-11-22 09:09:50 +00:00
dan 841ed2b0f2 Merge latest trunk changes with this branch.
FossilOrigin-Name: 7ef44c5b5bd30bcc4ef59ed172b9ce9ac6a843f6
2014-11-21 14:37:24 +00:00
dan 54f0dec3e6 Changes to comments in sqlite3ota.h.
FossilOrigin-Name: 14139542b68fbf01632a1b149cd6fca4bb01efa6
2014-11-21 11:22:11 +00:00
dan ca5ceb4229 Add support for updating virtual tables via ota.
FossilOrigin-Name: 4dfcfe543945aa60a7ac397a3bdb0ac9e20ef7b6
2014-11-21 10:46:23 +00:00
dan f8bd49ee22 Add the "ota_delta()" feature for delta-compressed updates.
FossilOrigin-Name: c64dcd1788f5cc7db197a0ec4ab0981f34a72c6b
2014-11-20 19:19:02 +00:00
dan 268c0f8844 Update the ota extension so that it can be used to update tables with external PRIMARY KEY indexes.
FossilOrigin-Name: 55066a1171cbd3077f5e6c8ceb2745e810d9476e
2014-11-20 17:37:08 +00:00
dan 54ef517012 Updates to support zipvfs in pass-through mode.
FossilOrigin-Name: 556c3de53ad33d11d33ec794345c2100aa76f3e1
2014-11-20 15:11:12 +00:00
dan 8a1e107811 Add tests for WITHOUT ROWID tables with composite primary keys.
FossilOrigin-Name: 712d413d29950b19d4afb18cfcf9d3afb302d0a0
2014-11-17 18:35:30 +00:00
dan d8260fd1e3 Fix a problem with the parameters to an OP_Affinity in one of the VM programs generated by sqlite3_index_writer() that was causing an OOB read.
FossilOrigin-Name: 447b33b34a9836992f5a8d50cd8647a10435c144
2014-11-17 15:07:40 +00:00
dan 48d7014067 Fix the customization interfaces so that they match the documentation.
FossilOrigin-Name: fba0b5fc7eead07a4853e78e02d788e7c714f6cd
2014-11-15 20:07:31 +00:00
drh 1728bcb07f Add the eval() SQL function extension in ext/misc/eval.c.
FossilOrigin-Name: 27cf665b957f2c0ced403e3032099e80c295598f
2014-11-10 16:49:56 +00:00
drh 051575cbf4 Do not use virtual (and hence redundant) WHERE-clause terms to restrict the
content of a automatic partial index.  Show when an automatic partial index
is used in the EXPLAIN QUERY PLAN output.

FossilOrigin-Name: b9ad601eab1d7298d369267eb697c7fa1bc16985
2014-10-25 12:28:25 +00:00
dan 08e06b0b21 Add tests to check error handling in OTA.
FossilOrigin-Name: ec7321ae482a8c4d893851a5edd17d67ef1a448b
2014-10-22 15:33:12 +00:00
dan d6446a0abe Test that sqlite3ota_open() works with URI paths. Fix some other issues.
FossilOrigin-Name: 6fd09854feb46739f42b7e7a5e76680d5f9b8c5a
2014-10-21 19:35:03 +00:00
dan 5e3fc6e99f Add tests for another application writing the database while an ota update is ongoing.
FossilOrigin-Name: 2402baa0027ca65e9a5106b8b9c4e10f40d2cbc3
2014-10-21 18:09:58 +00:00
dan ae0151c5a7 Merge version-3.8.7 changes with this branch.
FossilOrigin-Name: d380a6482a46478ebdf97e08b2fcf78b5d126dde
2014-10-20 16:34:03 +00:00
dan d2f5ee20f0 Have the ota extension perform an incremental checkpoint after generating the wal file.
FossilOrigin-Name: 0bf1301aacb3b717b4cc020fbda9fab0bae331c3
2014-10-20 16:24:23 +00:00
dan 6e1a037357 Allow FTS tokenizers to choose whether or not to consider the "*" character part of tokens or not. This restores the pre-[e21bf7a2ad] behaviour. Also fix a problem causing FTS to interpret tokens beginning with "*" characters as EOF.
FossilOrigin-Name: 49dfee7cd1c9ab2901b8a871a6cd00b2ead76801
2014-10-09 15:08:17 +00:00
drh a8950d5038 Fix a memory leak associated with the FTS4 matchinfo() function.
FossilOrigin-Name: fb8da82411b80a234c6a5481622027815450996a
2014-10-09 14:00:49 +00:00
drh b8e8d5055a Fix a harmless compiler warning inside an assert() in FTS4.
FossilOrigin-Name: 418f3c9ad28672e5fe38d772d34e7cf8d26bc0e1
2014-10-06 12:41:57 +00:00
drh 249489331c Add the "showauth" extension in ext/misc.
FossilOrigin-Name: 28d52c1c38d849f099bc777f5987d1ef89680c2a
2014-09-21 17:51:37 +00:00
dan 93fdbfa64f Add further tests to ota5.test. Add "ota.test", for running all ota tests.
FossilOrigin-Name: 95ffdaa542df1c28fac97422e5a4b2c5cb81d50a
2014-09-19 18:08:39 +00:00
dan d1193a9893 Add extra tests for the ota extension.
FossilOrigin-Name: 1e468fe1e408e513a1e1bbe72fe2a240f2991b3d
2014-09-19 15:06:23 +00:00
dan be758ce3ae Update the ota extension to support SQLITE_ENABLE_8_3_NAMES builds.
FossilOrigin-Name: 718da6de870231d358384473b40e81c4394b8067
2014-09-18 17:57:46 +00:00
dan 6c5984a950 Use quotes instead of angle-brackets to include sqlite3.h from sqlite3ota.h.
FossilOrigin-Name: fce9c6ccf101d28492a84b85463433f32ef569fb
2014-09-18 16:38:10 +00:00
dan 318e63c60b Remove some c++isms from sqlite3ota.c.
FossilOrigin-Name: 0da1862b1b68f15c7c14286a982dda886d1e3b4a
2014-09-18 15:57:22 +00:00
dan ed18c96107 Add new file ext/ota/README.txt, containing notes regarding the implementation of the ota extension.
FossilOrigin-Name: 3c6e1cbb4baaebc5958ab7276f27ff0ba14f4fa2
2014-09-18 15:22:48 +00:00
dan 4caa8fbc1e Add an API to query an ota handle for the total number of key/value operations performed so far.
FossilOrigin-Name: e3943fa7bbbfc5e16f73a494d8fa54d19e9cfcf9
2014-09-18 14:48:38 +00:00
dan c6c3d00f33 Fix an ota bug causing attempts to write to tables with more than 8 columns to fail.
FossilOrigin-Name: 7da98ca2422166dc5d68607541707f41b77b5784
2014-09-18 11:15:17 +00:00
dan 44ed6bc5c1 Fix an unintialized variable problem in sqlite3ota.c.
FossilOrigin-Name: 01268607515a30d07559648c43034e522605748a
2014-09-17 19:05:46 +00:00
dan abc0788663 Add tests and fixes for "PRAGMA ota_mode".
FossilOrigin-Name: 39df35c4ac65ffba76ba2c6f6727cf5e843e7517
2014-09-17 15:20:24 +00:00
dan 1b95de09bc Clarify the effects of the pager_ota_mode pragma. Add tests and fixes for the same.
FossilOrigin-Name: decaccc37cbdcd2a663233469efdf4982a810513
2014-09-16 20:02:41 +00:00
dan 15e1ec4c5b Remove the experimental sqlite3_transaction_save() and restore() APIs.
FossilOrigin-Name: 48d201cd8b68c0377cf8a2cc6439b893f9462fe2
2014-09-15 19:34:04 +00:00
dan d4a80e6742 Merge latest trunk changes with this branch.
FossilOrigin-Name: 55b8011d5b455927f5b92a3cb911fd909fb0edab
2014-09-15 15:34:31 +00:00
dan ee8d0b4111 Have sqlite3ota.c use grave accents instead of double-quotes to enclose identifiers in generated SQL. To avoid having the SQL engine substitute a literal string if a column reference cannot be resolved.
FossilOrigin-Name: 79f2418429aa05c56069c56d51b4d72f662a6970
2014-09-15 15:22:32 +00:00
dan 02cf6e1681 Ensure the correct collation sequences are used when sorting data in sqlite3ota.c.
FossilOrigin-Name: 473a72d7009a22ea514a98ee8869e7e7bca14cf5
2014-09-15 14:54:07 +00:00
dan 13ce7931f7 Have the sqlite3_index_writer() VMs check that the final values of records inserted into indexes on rowid tables are integers.
FossilOrigin-Name: cca376bff3510dc5e99fc5824862c3471ceced16
2014-09-15 12:18:29 +00:00
dan d54e06ca5e Add OP_Affinity opcodes to the VMs generated by sqlite3_index_writer().
FossilOrigin-Name: b9b38cb8e21826ddfb4c87a1f166ad33c3c7b6eb
2014-09-15 10:44:24 +00:00
drh c891c6c7ad Clean up some #includes in the extension API implementation.
FossilOrigin-Name: b149ef5c639e6bcff7bd1c7866353e7f7f468070
2014-09-11 17:14:54 +00:00
drh 7883ecfcd4 Enhance the sqlite3_user_add() interface to initialize the user
authentication logic.  Add test cases for the extra argument on the
end of the authorizer callback.

FossilOrigin-Name: 842c6da8f1a62bd13a1b4089a98b0835a46a2285
2014-09-11 16:19:31 +00:00
drh 9d5b0df132 Get the sqlite3_user_delete() interface working.
FossilOrigin-Name: 974a9c65583f7ab438d5673dc00c347ab8322855
2014-09-11 14:56:45 +00:00
drh 570f187f78 Fix the sqlite3_user_change() interface so that it does allow a
non-admin user to change their own password.

FossilOrigin-Name: 52d440c7e1b07fc03f14ed5fa4cc4c89a75cd430
2014-09-11 14:40:27 +00:00
drh 32c6a48b5e Add support for the extra parameter on the sqlite3_set_authorizer() callback
and support for failing an ATTACH with an authentication-required database
using bad credentials.  The extension is now feature complete, but much
testing and bug-fixing remains.

FossilOrigin-Name: 596e728b0eb19a34c888e33d4d37978ca2bf1e00
2014-09-11 13:44:52 +00:00
drh d39c40ff5e Reorder parameters on the sqlite3_user_*() interfaces for consistency.
Add the first TCL test cases.

FossilOrigin-Name: 2f6d8f32eef526b5912f42ab467e3c7812480d8b
2014-09-11 00:27:53 +00:00
drh 09e60541ae Complete the implementation of the various APIs. Fix several problems.
This is another incremental check-in that does not completely work.

FossilOrigin-Name: 4eaaa7fa87aa912d24f8b35440ab60310dc08310
2014-09-10 22:46:46 +00:00
drh f442e33e3a Add the ".user" shell command and implement the sqlite3_user_add()
routine.  Incremental check-in.  The code compiles but does not work.

FossilOrigin-Name: a0455f9deb603bf91684158d911269622720fc1a
2014-09-10 19:01:14 +00:00
drh e933b83f02 Further ideas on user authentication. Not yet working code.
FossilOrigin-Name: c8171ecd0d6f097c9e95d5f6643bae8d67f44750
2014-09-10 17:34:28 +00:00
drh d45309796b Non-working preliminary implementation attempts on user authentication.
FossilOrigin-Name: 8440f093bac19a41d44ee352744354eab897fe4e
2014-09-09 14:47:53 +00:00
dan a690402b4c Add support for update statements to sqlite3ota.c.
FossilOrigin-Name: e109b27e4d66b83e1a804e7556d9c91aa37fea28
2014-09-08 17:50:35 +00:00
dan 9295d21bfd Add support for delete operations to the ota extension.
FossilOrigin-Name: f988234ba54d7c667f7deef1d04beed4e7fe6182
2014-09-06 20:19:38 +00:00
dan 4e9246e9db Switch back to using a single database connection in sqlite3ota.c.
FossilOrigin-Name: 3c2f4a078132992e33cc675173c84f8385af9cb5
2014-09-05 19:52:42 +00:00
dan 0c4ba2662e Reorganize the code in sqlite3ota.c in preparation for adding support for update and delete operations.
FossilOrigin-Name: 98387f05697526c7740e91d8a846a31f77639406
2014-09-05 19:31:15 +00:00
dan 8fa7f590cc Avoid calling sqlite3OsFetch() on a file-handle for which the xFetch method is NULL.
FossilOrigin-Name: 071f7f2decd2f786c0201a4219e9c2cc9d227085
2014-09-04 11:03:35 +00:00
dan 19b465a0ae Split part of "PRAGMA ota_mode" off into "PRAGMA pager_ota_mode". This allows some specialized custom VFS implementations to intercept and implement the expected pager-related effects of this pragma.
FossilOrigin-Name: 209f672e588b54dfbfb83c7859cacdc4497f0f2b
2014-09-03 19:30:32 +00:00
dan 98231c054b Add a command line program that uses the extension. This serves as example code and is also useful for performance testing.
FossilOrigin-Name: ffa1524ef2a4c32652183eb4745685f0d1c93af2
2014-09-03 08:25:09 +00:00
dan b0083756f2 Add an experimental extension for applying bulk updates to databases.
FossilOrigin-Name: 2954ab501049968430011b63d046eb42ff37a56c
2014-09-02 19:59:40 +00:00
dan 4aee5dff83 Add documentation for tokenizer api to fts5.h. Also add a script to extract extension API docs and format them as html.
FossilOrigin-Name: e240d467e60b7755486aae5e8b0824f7c741f852
2014-08-25 19:58:54 +00:00
drh 5ab567078a Enhance the spellfix extension with the ability to specify a rowid when
inserting new rows.

FossilOrigin-Name: 369c480cda6fa66394b995346bbf51f3298446e1
2014-08-22 11:11:30 +00:00
drh b6b4b79f34 Fix a faulty assert() statement. Add comments to clarify the behavior of
the sqlite3OpenTableAndIndices() routine in insert.c.  Add test cases to
verify that the assert() statement is not firing inappropriately.
Ticket [369d57fb8e5ccdff06f1].

FossilOrigin-Name: 7029b3404d3f5f698a496934f3a3f2972051b257
2014-08-21 14:10:23 +00:00
dan 6885bbc713 Add an "automerge=0" mode that disables auto-merging and falls back to fts4-style crisis merges.
FossilOrigin-Name: 2397404e152b908d838e6491294b263b05943b3f
2014-08-18 19:30:01 +00:00
dan 0bbd9c27c2 Cache the value of the "totals" record in memory during transactions.
FossilOrigin-Name: 05dfdad445b22f375b71abe0b1fa1bf7ca331be7
2014-08-12 16:07:35 +00:00
dan e2fb318e34 Automatically resize the hash table used by fts5.
FossilOrigin-Name: f1cb48f412a5f200f1fe04f91072864f379db08f
2014-08-12 08:36:00 +00:00
dan 1a669f84a1 Simplify the way position lists are copied when merging data.
FossilOrigin-Name: 9f8d678a0ea75e169daf8b3f00bd05f52a050ea6
2014-08-11 20:26:34 +00:00
dan c7fe7a969b Replace the hash table borrowed from fts3.
FossilOrigin-Name: 617e2fac1c128212254f71b1a8fddaf0d1d90262
2014-08-11 19:44:52 +00:00
drh 490fe86f1a Fix harmless compiler warnings.
FossilOrigin-Name: 52b03f045edf6fc29f9ba9a5cac53a59f0feb0eb
2014-08-11 14:21:32 +00:00
dan c45f24b281 Fix an uninitialized variable causing a problem during fts5 table initialization.
FossilOrigin-Name: a14fa876f0eb66028e302b908967cc4a05ede9fc
2014-08-09 18:22:59 +00:00
dan b191db8390 Use multiple memory allocations for a single Fts5Structure object. This is probably less efficient but much easier to get right.
FossilOrigin-Name: 2821825f7a481755a333dcdcad780b3e24448f20
2014-08-09 18:02:27 +00:00
dan 1d3ed1f46a Add "segment promotion" to fts5. This prevents the FTS index from growing indefinitely as data is added and deleted.
FossilOrigin-Name: ba359d78e166d78e0dc89e3c63a9a41e9ffea989
2014-08-07 18:47:33 +00:00
dan d90aab8f94 Avoid writing delete markers to the oldest segment in an FTS index.
FossilOrigin-Name: 1baeb1cee61d9c56c718b50af034a24f1018a322
2014-08-06 20:04:14 +00:00
drh 858b638d1f A couple more harmless compiler warnings eliminated.
FossilOrigin-Name: bcf6d775f90f4d1ba018a1b965f2f710df130f01
2014-08-06 18:50:51 +00:00
drh e8f2c9dc71 Fix two more harmless compiler warnings. Make sure the fts3_unicode2.c file
is in sync with mkunicode.tcl.

FossilOrigin-Name: a2a60307ea68a3230952a56cb65369ba0a208967
2014-08-06 17:49:13 +00:00
dan 7b71fbaf93 Add support for savepoints to fts5.
FossilOrigin-Name: 3b19eba042bb2eeb1be60f8d58ebaa0a045d6a5c
2014-08-06 16:30:21 +00:00
dan 192d413e95 Use doclist indexes for AND queries as well as phrases.
FossilOrigin-Name: 5d38e6edc40ef188fbf96505073797036aa6783a
2014-08-05 19:35:20 +00:00
dan d5997d80cb Use doclist-indexes with "ORDER BY rowid ASC" fts5 queries as well.
FossilOrigin-Name: d028ba6589f3122b635474c2683c0f93d5bc6c7c
2014-08-05 19:00:22 +00:00
dan 9a67058a63 Fix fts5_index.c to use doclist-indexes when possible. Only some cases work so far.
FossilOrigin-Name: 90b82d3ef613b2915e0e280dc1d2e5a2b617d59c
2014-08-04 20:07:40 +00:00
dan 9af0705e84 Start changing things to use doclist indexes as required. code is not activated yet.
FossilOrigin-Name: b8864da95db2c0e611116304d607e35a86c9247d
2014-08-02 20:49:36 +00:00
dan ff31da03e5 Add a special case to the integrity-check code to check that the final integer in a doclist index is as expected.
FossilOrigin-Name: c98934155cb48adfda57bd0fd1b950226d45f67a
2014-08-01 20:13:49 +00:00
dan d34742901d Have the fts5 integrity-check verify that doclist indexes match the contents of the leaf pages that they index.
FossilOrigin-Name: 37a7d3035eb4bbad7e32fe550321ac9fae611a57
2014-08-01 19:27:07 +00:00
dan a29284d65f Add "doclist index" records to the database. These are to make navigating within very large doclists faster. They are not yet used by queries.
FossilOrigin-Name: 89377421ff69f2450364987afe781b6d8bcbf087
2014-08-01 11:16:25 +00:00
dan 3dbfc8d8e8 Add a comment explaining why fts5 cannot cache "sorter statements".
FossilOrigin-Name: e6af3b7a3cf331210f4c87848e2af007dbd5ef30
2014-07-31 17:53:03 +00:00
dan 937490d4b4 Add further tests for the extension APIs with "ORDER BY rank" queries.
FossilOrigin-Name: 37a417d27e4ebafd4783f62728d7467316b75b17
2014-07-31 11:57:59 +00:00
dan 3fc4a856a2 Fix things so that the fts5 extension API works with "ORDER BY rank" queries.
FossilOrigin-Name: f1b4e1a98d49ecaba962beba16f8224175e4ba59
2014-07-30 20:26:24 +00:00
dan fdb4a30af2 Add hidden column "rank". Currently this always returns the same value as the bm25() function.
FossilOrigin-Name: 4cc048c3651e830a6aeded924c7f3a60b634e133
2014-07-30 19:41:58 +00:00
dan 57ff60b19b Have calls to the xFilter() method of rtree virtual tables ensure that cursor is initialized before proceeding. Fix for [d2889096e7bdeac].
FossilOrigin-Name: 8cc41b0bf365af47c2061ffe44c86018945dd239
2014-07-29 11:54:18 +00:00
dan 92e497e517 Add the "loadfts" program, for performance testing the loading of data into fts3/fts4/fts5 tables.
FossilOrigin-Name: 770b9540c19ad1e3d24adff382332bf032065efd
2014-07-28 20:14:02 +00:00
dan 454b5ce524 Add tests and fixes for bm25() function.
FossilOrigin-Name: 71d32f53e81921e43c933cc968cb1c18d83fe1e0
2014-07-26 18:38:51 +00:00
dan 700b33d7a5 Add extension apis xRowCount, xQueryPhrase, xSetAuxdata and xGetAuxdata. And a ranking function that uses all of the above.
FossilOrigin-Name: c4d50428ab97f77e6721c4f8d03eaaf3ea91f3eb
2014-07-25 20:30:47 +00:00
dan b30860abe2 Add a snippet() function to fts5.
FossilOrigin-Name: bdc58fd28a63ac9632c3df6c7768a9a236566605
2014-07-23 19:31:56 +00:00
drh f5d87f77e2 Fix the index name for the shadow tables in the spellfix1 extension so that
multiple instances of the spellfix1 virtual table can each have their own
index.

FossilOrigin-Name: 438c348a40383796a710499e4e1b7a6b08f75778
2014-07-22 16:00:49 +00:00