Commit Graph

1300 Commits

Author SHA1 Message Date
drh 4977ccf143 Fix an off-by-one error (really off-by-2 in this case) in the buffer
resize logic of json1.

FossilOrigin-Name: d2a027372a5a6efc0f9b6f605093d865ae1c6788
2015-09-19 11:57:26 +00:00
dan 2e8f5517f7 Avoid passing (signed char) values directly to isspace(), isalnum() or isdigit() in json1.c. Cast the value to (unsigned char) first.
FossilOrigin-Name: 6713e35b8a8c997aa2717e86ce6dcd63bb993477
2015-09-17 17:21:09 +00:00
dan cf346d7a9c Fix a crash in FTS5 caused by specifying NULL as the rhs of a MATCH operator.
FossilOrigin-Name: e8ed62f82e8acc40b818bf86fafe3d480687514e
2015-09-17 09:48:01 +00:00
dan ec9e8f004e Fix typo in fts5.h.
FossilOrigin-Name: 07721c6c24f651d8e281c29227b113d292f6b0de
2015-09-16 12:31:39 +00:00
dan 6609bab24d Add test cases to cover fts5 integrity-check code.
FossilOrigin-Name: 1d018c35b9e81982df036f5e62a4a42219b54e02
2015-09-15 14:39:17 +00:00
dan 128ea696ea Fix a problem with fts5 "ORDER BY rowid DESC" queries and large terms.
FossilOrigin-Name: b26d8f79c62cb71b4c3ea5df9d0e09558bbbd794
2015-09-15 11:58:33 +00:00
dan dafeda754b Improve test coverage of fts5_index.c.
FossilOrigin-Name: c1f76686cee3918b1be785a4071d68cb3afda0ef
2015-09-14 19:51:05 +00:00
mistachkin 16a93128f8 Fix harmless compiler warnings.
FossilOrigin-Name: bfc7b84b766860d2e410702ba7c1166d7328309a
2015-09-11 18:05:01 +00:00
dan bcb04b6128 Fix some compiler warnings in fts5 code.
FossilOrigin-Name: 0dc436116e55e3fd55eb6085ada71e099069b32d
2015-09-11 14:15:46 +00:00
drh f5ddb9c214 Take out the goofy '$$' path syntax. Instead, use subtypes to communicate when
a string is JSON.  Add the json() function that validates and minifies the
JSON and sets the appropriate subtype.

FossilOrigin-Name: 8a80d6459e246ec1b38325e1cbd1e862157138b3
2015-09-11 00:06:41 +00:00
dan 33c47f565c Modify the fts5 leaf page format to permit faster seek operations. This is a file-format change. Any existing databases can be upgraded by running the fts5 'rebuild' command.
FossilOrigin-Name: 0c0c4ae971e54efc526eed7bd071c90dfadb95ff
2015-09-10 17:23:37 +00:00
drh 383de6918c Create separate "path" and "root" columns in the json_each() and json_tree()
virtual tables.  "Root" is the 2nd parameter and is fixed.  
"Path" varies as json_tree() walks the hierarchy.

FossilOrigin-Name: 127cce3eb96b819005832997e0a082df9fb96f0b
2015-09-10 17:20:57 +00:00
dan 204debf3a3 Increment the fts5 version value to indicate that the on-disk format has changed.
FossilOrigin-Name: 99de5e3613d557728dd196353516bc7cf64a0e6c
2015-09-10 16:39:38 +00:00
dan 56c8634853 Fix a segfault in fts5 that could occur if the database contents were corrupt.
FossilOrigin-Name: 4931e37da4d2c26d7afc5432f7f0d534b51a85fa
2015-09-10 16:19:01 +00:00
dan 40d45f8960 Merge latest changes from trunk. Including fts5_expr.c fixes.
FossilOrigin-Name: 716e7e747714d6af502f6a87ca8d789bb7ce162a
2015-09-10 15:52:42 +00:00
dan b97efa0ba3 Update description of on-disk format in fts5_index.c.
FossilOrigin-Name: 85aac7b8b6731e2f6880b80cfd62d431ea059799
2015-09-10 15:49:16 +00:00
dan f679d97d18 Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages.
FossilOrigin-Name: ffe2796ac9244c62325fce4960f26c653321623c
2015-09-10 10:01:30 +00:00
dan a5aa8e1db2 Change the array of 16-bit offsets at the end of each page to an array of varints.
FossilOrigin-Name: fab245bea4f283714c17bca22428d5eb4db5935a
2015-09-10 05:40:17 +00:00
drh 852944eb9b Fix the json_tree() scan for the case when a path is supplied. Add new
json1 test cases.

FossilOrigin-Name: 6adc7de76acee6cfb5ff761739e7a8de7b5bf4b2
2015-09-10 03:29:11 +00:00
mistachkin f5eac36a64 Fix harmless compiler warning in FTS5.
FossilOrigin-Name: 86146a731d75eb25279c0e072c0bdda593de905d
2015-09-09 23:54:46 +00:00
mistachkin 4b03efb695 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: 2cdb18778f20baa902e54f218ba26d2bacf0801e
2015-09-09 17:17:22 +00:00
dan f06bd2c132 Fix a bug in preprocessor macros within fts5_main.c.
FossilOrigin-Name: 0eb2b9521fad6fa36e6fa374c2bc1f70b5180f7c
2015-09-09 08:15:06 +00:00
dan fe8e2eba0a Remove the 0x00 terminators from the end of doclists stored on disk.
FossilOrigin-Name: 00d990061dec3661b0376bd167082942d5563bfe
2015-09-08 19:55:26 +00:00
dan 60f8139a96 Use macros to make the code in fts5_index.c easier to read.
FossilOrigin-Name: 67ff5ae81357eb7fa28049bb724a22cb6f52e076
2015-09-07 08:14:30 +00:00
dan e386a1ba25 Experiment with a different fts5 leaf page format that allows faster seeks.
FossilOrigin-Name: a1f4c3b543eed84e808f6b901a38179786fffe16
2015-09-05 19:52:08 +00:00
dan 4dfe98a8c4 Enhance showfts5.tcl so that it can optionally display the number of terms in each segment.
FossilOrigin-Name: d648ddd93de039820f5abe064c7bc1318cd9d6b1
2015-09-04 11:13:00 +00:00
dan f8efe64649 Rearrange code in fts5_expr.c so that synonym support does not slow down the common case.
FossilOrigin-Name: 801882817f1d895aef1426f9a7196bd140b807c3
2015-09-03 19:48:02 +00:00
dan cce413202a Merge enhancements from trunk.
FossilOrigin-Name: 4b49fe996989fe42d2bb1c24e7193fef09c5fc50
2015-09-03 18:57:52 +00:00
dan ea8b93ee20 Add documentation for fts5 synonym support.
FossilOrigin-Name: 58aa1f435959852df74f1bca8e0bdbc4f47c256a
2015-09-03 18:05:09 +00:00
dan ea623630e2 Add tests to improve coverage of fts5_varint.c.
FossilOrigin-Name: 89f24f31a8f7d7cb0a66ee53523881f566dcb035
2015-09-03 15:37:26 +00:00
dan e1c28d7229 Remove some more code from fts5_index.c by consolidating similar functions.
FossilOrigin-Name: 59ae30b97b40faa363c55aa2664dead9eaeeddc0
2015-09-03 14:22:27 +00:00
dan 5d144875bc Remove dead code from fts5_index.c.
FossilOrigin-Name: 8a0a9b01e74072ee52fe393311ad591208fbbf7c
2015-09-03 11:17:52 +00:00
dan 9a9e3531f7 Fix the fts5 integrity-check so that it works with columnsize=0 tables.
FossilOrigin-Name: 11b887b15eaee57ea2577c763e70494f1e251275
2015-09-03 11:00:50 +00:00
dan 2a6ecb3141 Fix a memory leak in fts5_expr.c.
FossilOrigin-Name: 399932a181437d84cd57234e76c4b65da1e4ac5f
2015-09-03 10:27:02 +00:00
dan 9c671b741c Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same.
FossilOrigin-Name: bdedd838bb3028c586bcc9f643852ce1364adb49
2015-09-02 19:48:55 +00:00
dan 50ea76e748 Fix an issue with fts5 synonyms and NEAR(...) queries.
FossilOrigin-Name: f2e590700d8eb30da8ac3a92f6ca92a2bbe9c1fe
2015-09-02 18:56:01 +00:00
dan bea34fc53a Fix a problem with fts5 synonyms and phrase queries. Also fix an OOM handling bug in fts5.
FossilOrigin-Name: a4c35fa2c94fe34b376670244fe72303c99868c1
2015-09-02 17:34:22 +00:00
dan 583611df65 Fix a problem handling OOM conditions within fts5 queries that feature synonyms.
FossilOrigin-Name: 11fa980897c6c7be218bbd9c4cd8253272d2c300
2015-09-02 14:17:38 +00:00
dan df998c3d4f Fix a problem with fts5 synonyms and the xQueryPhrase() auxiliary function API.
FossilOrigin-Name: cf3e45e76d23e10ee06296c3561a341591597a04
2015-09-02 08:22:41 +00:00
dan d917ad019c Fix a problem that occurs when more than 4 synonyms for a term appear within a single row.
FossilOrigin-Name: cd359550bdc2bf7be4c294b60130c9fc3f583902
2015-09-01 18:44:33 +00:00
dan 3faa07ee04 Add tests for fts5 synonyms implemented by adding extra terms to queries. And fixes for the same.
FossilOrigin-Name: dbcb73802b88f76be17f09f3eb83ffac542de633
2015-09-01 18:08:36 +00:00
dan cb8b58caf7 Begin changes to allow synonym support by adding multiple terms to a query (an alternative to adding multiple terms to the FTS index).
FossilOrigin-Name: ad7feaed4cd6b1d6e6376bb82d1f5664ddd083f3
2015-08-31 20:06:06 +00:00
drh 3ad93bba84 Enhance the json_extract() function so that if given multiple PATH arguments
it will return a JSON array with all of the answers.  Also update comments
within the json1 extension to reflect stricter interpretation of JSON and PATH
arguments.

FossilOrigin-Name: 1da60c3dda4254620052a83c853c2d2b6dd5009f
2015-08-29 19:41:45 +00:00
dan 0cb01f5c2c Add a test for an fts5 tokenizer that supports synonyms by adding multiple entries to the fts index.
FossilOrigin-Name: 98d07d16cab92f1e7001afbe370df3ec6343fc1f
2015-08-29 18:46:12 +00:00
drh d29759288e Fix the build with -DSQLITE_OMIT_VIRTUALTABLE.
FossilOrigin-Name: 752918def7231f7846b3e985c9953a1cc825ab6b
2015-08-29 17:22:33 +00:00
drh d1f0068d19 Do not consider an empty string to be valid JSON. Add some additional
JSON test cases.

FossilOrigin-Name: fd19ff029f128f478f69910352a6f8b84262ce1d
2015-08-29 16:02:37 +00:00
dan ee0c0a8de3 Another change to the fts5 tokenizer API.
FossilOrigin-Name: fc71868496f45f9c7a79ed2bf2d164a7c4718ce1
2015-08-29 15:44:27 +00:00
drh a771402e55 Change the json1.c module so that it throws an error if any of the
JSON selector paths are malformed.

FossilOrigin-Name: 3aa0855fd463076fc3277f1d9fe00d2f30e6b449
2015-08-29 00:54:49 +00:00
drh f2df7e71d6 Back out the json_check() routine. Instead, throw an error if the input to
a json function (other than json_valid()) is not valid JSON.

FossilOrigin-Name: dc9ce7b18cbe23d065317757234ef9fb8792da7a
2015-08-28 20:07:40 +00:00
dan 57e0add3f9 Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document.
FossilOrigin-Name: 90b85b42f2b2dd3e939b129b7df2b822a05e243d
2015-08-28 19:56:47 +00:00
dan 2798f0b54b Fix compiler warnings in rbu code.
FossilOrigin-Name: 0fdc36fe35ae2fc8e9688fe6c53437f4d47502d9
2015-08-28 16:41:45 +00:00
drh f6ec8d4f4c Add the json_check() function, which returns its argument if the argument
is well-formed JSON or which throws an error otherwise.

FossilOrigin-Name: 64abb65d4df11e5b3bcc4afc8e7c18e907c6080a
2015-08-28 03:48:04 +00:00
drh ecb5fedb3f Enhance the json_insert(), json_replace(), and json_set() functions with the
ability to add JSON instead of text if the argument is text and if the PATH
begins with '$$' instead of just '$'.

FossilOrigin-Name: 44f103d8862abc2d5613bac04dc2ea8e625b1f40
2015-08-28 03:33:50 +00:00
drh 80d874083b Improvements to JSON string dequoting.
FossilOrigin-Name: 196d66d34d9783622e6f2f79eafea1488fc6f5cf
2015-08-24 12:42:41 +00:00
drh 442a7c6015 Fix corner-case problems in the type and atom columns of json_each() and
json_tree().

FossilOrigin-Name: f0aba0e120074430cd7ad93291fcc97b8a25a054
2015-08-24 02:32:04 +00:00
drh c3722b2103 Fix minor glitches in the json1.c extension, mostly having to do with OOM
behavior.

FossilOrigin-Name: cc5204149c4053b9e529a72102d8df0925ad1ea1
2015-08-23 20:44:59 +00:00
drh 8784eca17f Fixes to json_each() and json_tree(). Improved json_parse() debugging output.
FossilOrigin-Name: fc1b24f316af07a64672f6edc14ebcff487dffbb
2015-08-23 02:42:30 +00:00
drh bc8f092ca1 Add the json_valid() function to the json1.c extension. Fix various minor
problems in the json1.c extension.

FossilOrigin-Name: 380a97345b446214843a63ccc017d49a52d884da
2015-08-22 19:39:04 +00:00
drh be9474ee05 Fix a couple instances of OOM handling in the json extension.
FossilOrigin-Name: 213a6c5ccbcfe4495d45e3608e99a6398751aeed
2015-08-22 03:05:54 +00:00
drh 6fd5c1e0e8 Fixes for compiler warnings and errors in the makefiles. Rename the
one test script to json101.test.

FossilOrigin-Name: 9ff6ccde5f26f18073587c320290570854ffc833
2015-08-21 20:37:12 +00:00
drh f28ed795db Change the name of the json loadable extension to "json1.c", in anticipation
of having future major changes to the interface.

FossilOrigin-Name: d0d4bec9e3d8829a2d488f2742f1650214fa716a
2015-08-21 20:12:43 +00:00
drh 4af352d47e Add the fullkey column to both json_each() and json_tree().
FossilOrigin-Name: 15dd99431e9ddd0fbdbb8dcc921687b0c6d26a29
2015-08-21 20:02:48 +00:00
drh 505ad2ca41 Add the json_tree() virtual table.
FossilOrigin-Name: 08c36e45f0d3a7b89caf823652d7543b76ac802a
2015-08-21 17:33:11 +00:00
drh a7781dc0bb Merge in trunk fixes for table-valued functions.
FossilOrigin-Name: 67375f32d931351964d75d20671c4a6d8a294d7d
2015-08-21 17:16:22 +00:00
drh 1f2fc28134 Fix a corner-case bug in table-valued functions. Update the generate_series()
virtual table to increase the performance estimate penalty for being
underspecified.

FossilOrigin-Name: 552bc9cb88bbe54b4cf5fdf66d1217e7a2047110
2015-08-21 17:14:48 +00:00
drh 6842952429 Merge the latest changes from trunk, including the table-valued function
implementation.

FossilOrigin-Name: 10c444322f05ea8b83247f96a140bf3dbc4e9448
2015-08-20 23:39:38 +00:00
drh d8b1bfc6bf Fix corner-case memory management issues in table-valued functions. Change
virtual table handling so that if xDestroy is missing the table is
eponymous only even if xCreate is present.

FossilOrigin-Name: 774e6a14b124bbae4da0e188b62aee9ffb8c3745
2015-08-20 23:21:34 +00:00
drh b81fba1930 Prevent the series.c extension from loading on older versions of SQLite that
do not support xCreate==NULL.

FossilOrigin-Name: 3efc79427ef4686142d074cfe5b2f0a33af19b2e
2015-08-20 18:28:46 +00:00
dan 01dced109f Fix a typo in series.c.
FossilOrigin-Name: 23db7f50f14801c2cf56c006d7c7f593908b7158
2015-08-20 16:16:37 +00:00
drh 6b43cc8531 Fix the path lookup for objects so that it can handle quoted identifier
names and non-alphanumerics in the identifier.

FossilOrigin-Name: 87f5873004f69396baa7c67937342b4e32f79bda
2015-08-19 23:02:49 +00:00
drh cb6c6c6f45 Add the json_each(JSON,PATH) table-valued-function.
FossilOrigin-Name: 3335ac17bbcb09dc915173d69bf42048f84ad563
2015-08-19 22:47:17 +00:00
drh 222cd5bee2 Merge the table-valued-function rowid fix.
FossilOrigin-Name: a06a6392bd48baa8b9bac2624869c0cc7da7e779
2015-08-19 19:26:13 +00:00
drh 509c3fc004 Fix eponymous virtual tables so that they do not automatically make the first
column the rowid.  Enhance the generate_series virtual table to support rowid.

FossilOrigin-Name: a325a08599759471047e234ef9cfcc3cb110aafd
2015-08-19 19:01:28 +00:00
drh 03726aaf7f Merge support for table-valued functions.
FossilOrigin-Name: 96a5d44d9fcb7b159c04630ad2d956fe27df5a43
2015-08-19 18:31:11 +00:00
drh bc550df32f Improved comments on the generate_series virtual table. Test cases for
ORDER BY rowid DESC with generate_series.

FossilOrigin-Name: fef44c37f31ca9fd7891cecdbe95cc46a987067b
2015-08-19 18:19:49 +00:00
drh 398f872d1f Virtual table modules with a null xCreate method act as eponymous-only modules -
they cannot be used in a CREATE VIRTUAL TABLE statement.  Add the series.c
extension that implements a postgres-like generate_series virtual table to
demonstrate this capability.

FossilOrigin-Name: c58426dbd5ea8b8440ebcc1214f79fa63d658216
2015-08-19 13:54:20 +00:00
dan 5ef2b281aa When updating a zipvfs database with RBU, set journal_mode=off to prevent the upper-level pager from creating a large temporary file.
FossilOrigin-Name: dec14a3980dcf6c61b0b9ac4ba82fa963ac346d4
2015-08-19 08:34:34 +00:00
mistachkin 0cbb513a67 Fix harmless compiler warning in FTS5.
FossilOrigin-Name: 02448a9fccd4532daa667e4ba6ff1dbb7f68008a
2015-08-18 16:32:45 +00:00
drh 0042a9785c Comment clarification. No changes to code.
FossilOrigin-Name: 71a966952cf8f1d699cf8362ea3f6204c6e16384
2015-08-18 12:59:58 +00:00
drh 52216adf7e Initial implementation of json_set() and json_insert().
FossilOrigin-Name: 4aa49656d98e2894f2faa8963f79462ee6165d40
2015-08-18 02:28:03 +00:00
drh d096059fca Initial implementation of json_replace().
FossilOrigin-Name: 3c4bee65d93efc7f03f0f11817a068b02068d37c
2015-08-17 21:22:32 +00:00
drh 301eeccaa7 Add an initial implementation for json_remove().
FossilOrigin-Name: 2a8267209dbda36a37c1b5f65000b2f763c62341
2015-08-17 20:14:19 +00:00
drh 987eb1fa9b Initial implementation for json_array_length(), json_extract(), and
json_type().

FossilOrigin-Name: 39983204515837e7bd574cf47918e493acc03d1f
2015-08-17 15:17:37 +00:00
drh 5634cc05e4 Abandon the JSONB format for now. (We may return to it in the future.) Add
a function to render a JSON parse.

FossilOrigin-Name: 9703c0aa18ae43375af876474b818e504e1c10a5
2015-08-17 11:28:03 +00:00
drh b2cd10eaa9 Fix off-by-one error when parsing primitive JSON types "true", "false", and
"null".

FossilOrigin-Name: 42c15c1e36b5077646fef99028cf12e587a45023
2015-08-15 21:29:14 +00:00
drh e9c37f32e5 Add a prototype JSON parser.
FossilOrigin-Name: 789ba487000aa73621a41d115ad5de455ea8ea31
2015-08-15 21:25:36 +00:00
dan 82d7aea6d4 Add the sqlite3rbu_savestate() function to the rbu extension. To force rbu to save its state to disk without closing the sqlite3rbu* handle.
FossilOrigin-Name: 851a875ad6b81f90960caf4d03b116afc911858d
2015-08-13 18:26:09 +00:00
drh bd0621b360 Experimental code (untested) for a JSONB datatype.
FossilOrigin-Name: e3596ac7b1dd5bde3f9cae5781a6806d8d9f7166
2015-08-13 13:54:59 +00:00
dan 4510faa6df Avoid invoking system call unlink() directly from RBU code. Use the xDelete method of the default VFS instead.
FossilOrigin-Name: ee966af8ff79189f8b90f3557dea21059517dfe2
2015-08-13 11:46:21 +00:00
drh 2032d60793 Add the json_object() function.
FossilOrigin-Name: 414a95f3b79359f167858b2325fd2be223569c66
2015-08-12 17:23:34 +00:00
drh 5fa5c10784 Begin adding an extension that provides JSON SQL functions.
FossilOrigin-Name: dde8afdd8dba1d92560326dca7c1cdfedbe5e070
2015-08-12 16:49:40 +00:00
dan 7899fa78c1 Minor optimization for fts5 API xInst().
FossilOrigin-Name: efb7c9c5d0015c8c966f8d6742c05cda82fc146a
2015-08-12 15:36:58 +00:00
dan 7918dc835b Add the xPhraseFirst() and xPhraseNext() fts5 APIs, for faster iteration through a single phrases position list. Also optimize xInst() and xInstCount() a bit.
FossilOrigin-Name: f7682435278419829a46bb4cc9b5625d46549e22
2015-08-12 12:11:28 +00:00
dan f053ceb9c1 Merge fixes from the fts5NoWarn branch.
FossilOrigin-Name: 61cb2fc6c12810863c965c74e90bc502e20cf810
2015-08-11 14:25:34 +00:00
drh 027616d4ce Fix compiler warnings and remove unreachable code.
FossilOrigin-Name: be190fe7826d157fc5acd84210095725b059e4f6
2015-08-08 22:47:47 +00:00
dan d50bd6b0aa Update RBU to avoid repreparing a statement immediately after it is prepared.
FossilOrigin-Name: 1d75a41bb2f7fcd32f9f08768517b814e452e456
2015-08-08 15:13:08 +00:00
mistachkin 7e968a7d0a Fix harmless compiler warnings in FTS5 code.
FossilOrigin-Name: 0ddb2532b2daaaf1b0109ac360822f84cb999b7f
2015-08-06 03:19:14 +00:00
mistachkin c38e00f61e Fix duplicate test numbering in the FTS5 matchinfo tests.
FossilOrigin-Name: 483ebe8989ed7dbe197fe84e3eef6e76fa828e14
2015-08-06 03:09:49 +00:00
dan 50b5491771 Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so.
FossilOrigin-Name: 9e3aafe44a0813aa2a0c6172fdba1440b8a973ec
2015-08-05 19:35:59 +00:00
dan b20a42e316 Update the spellfix virtual table extension so that an explicit "top = ?" constraint works even if there is also a "distance < ?" or "distance <= ?" constraint.
FossilOrigin-Name: 0888838371608558f31d5bcb5fed1c8861aa52c1
2015-08-05 15:29:32 +00:00
dan dca929040a Fix some problems in test code detected by cppcheck.
FossilOrigin-Name: c9ff4ca4a0b44bbb8d6676144ddf77b04426e619
2015-08-05 08:01:46 +00:00
dan f9419d17ff Remove all references to "docid" within fts5 source code and comments. Replace with "rowid".
FossilOrigin-Name: dffd358f6cbf575d3b1045b1ce53429d15bade2a
2015-08-05 07:43:46 +00:00
dan d3789c0028 Add new test file fts5_test_mi.c, containing an implementation of a function similar to FTS4 matchinfo() for FTS5.
FossilOrigin-Name: 4f9520a9dc9c667b7fda5b0822de2bf48184ac99
2015-08-04 20:29:00 +00:00
dan 0734b5a057 Add extra tests for RBU and FTS3/4.
FossilOrigin-Name: 3419044967258e3ed65f71528e06952ee102dcc7
2015-08-01 18:18:21 +00:00
dan a9ca8afef5 Add support for "fossil deltas" to RBU and "sqldiff --rbu".
FossilOrigin-Name: e26ef165fe2f7524684af0d269d38475ea8b9489
2015-07-31 19:52:03 +00:00
dan 90c3179bb4 Remove a duplicate "typedef struct Fts5Global Fts5Global;" from fts5_main.c (duplicate is in fts5Int.h).
FossilOrigin-Name: 54a771fe2c2b3c1cc2914a527a825934fe114c49
2015-07-31 17:48:07 +00:00
dan dd688e7970 Fix a problem causing [sqldiff --rbu] to fail on tables for which all columns are part of the PRIMARY KEY.
FossilOrigin-Name: 93449e7046d60cad020ca439ded82e759c2e3cd9
2015-07-31 15:13:29 +00:00
dan 79e2347fdf Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer.
FossilOrigin-Name: c3c672af97edf2ae5d793f6fa47364370aa4f4ec
2015-07-31 14:43:02 +00:00
dan 994618521c Add the --rbu switch to the "sqldiff" utility.
FossilOrigin-Name: 098bea26da4533d9ad97a85687cca56fb0d764a4
2015-07-30 20:26:16 +00:00
dan c1e8ae679a Allow RBU tables to be named "data[0-9]*_<target>" instead of strictly "data_<target>". Also update RBU so that it always processes data tables in order sorted by name.
FossilOrigin-Name: 287aa30601506f168d355c35176a3383474444ca
2015-07-30 11:38:19 +00:00
dan dbac3ece0f Return an error message (instead of segfaulting) if test function fts5_expr() is invoked with no arguments.
FossilOrigin-Name: 8e818b8985c0196cd9671a6491796faaeebeb16e
2015-07-30 11:26:10 +00:00
dan 0f43df31b5 Update comments in fts5.h describing the xColumnCount() API function.
FossilOrigin-Name: a623eb55d1583bcfe360a8dabd29f2a583c61e64
2015-07-29 14:58:47 +00:00
dan 291d090947 Merge the fts5-btree-index branch with this one. FTS5 changes only.
FossilOrigin-Name: 9f1dcdc3e1644c21762dacf619fac70afe6318c5
2015-07-27 12:19:11 +00:00
dan c3b4132f53 Bump the "version" field in the fts5 config table.
FossilOrigin-Name: 92c9178933ef95e92c343c0a741a76bbdf071966
2015-07-27 11:49:01 +00:00
dan 8d90087146 Minor tweak to the SQL schema used by fts5.
FossilOrigin-Name: 427bf09ba41158041d774aac2b5ffbd062c6eabe
2015-07-27 11:43:36 +00:00
dan d0dc8cb5a3 Remove a potentially undefined behaviour involving signed integers and bitshift operations from fts5_hash.c.
FossilOrigin-Name: ad6286ab1f40e4716456a67b719f9cd733f988eb
2015-07-27 11:01:19 +00:00
dan 44198f30b5 Merge latest trunk changes with this branch.
FossilOrigin-Name: 5ec933c257884019484db7f533ce920012e47a1c
2015-07-27 10:46:48 +00:00
dan 5a51c272ea Remove a duplicated "if" condition from fts3.c.
FossilOrigin-Name: ee348b12754abda1fe13231c1868faca9d78481c
2015-07-25 19:26:52 +00:00
mistachkin 24fa5435ee Enable the RBU extension to compile cleanly on Windows using MSVC.
FossilOrigin-Name: cca79fdc3dff65907c2a59369057265b4512058f
2015-07-24 21:19:25 +00:00
dan 08406bf064 Fix warnings in fts5 code.
FossilOrigin-Name: bd4f156d07d0db1f063d42891965920fc2229352
2015-07-24 20:34:40 +00:00
dan 5555741bdc Allow the RBU module to read data from appropriately named SQL views created within the RBU database.
FossilOrigin-Name: 45c6a760ca63d19a7ccc352c7e35d8391025b515
2015-07-24 18:58:59 +00:00
mistachkin f6972c3764 Update RBU description comment in the header file. No changes to code.
FossilOrigin-Name: 119883a21bdcab66c15d1e80f3da20addda328ad
2015-07-24 00:02:15 +00:00
drh cfb8f8d6a9 Change the name of the OTA extension to RBU: Resumable Bulk Update.
FossilOrigin-Name: 017c5019e1ce042025d4f327e50ec50af49f9fa4
2015-07-23 20:44:49 +00:00
dan 3baaacae03 Merge trunk changes, including fixes for compiler warnings in fts5 code, with this branch.
FossilOrigin-Name: 7190d79ba452ceb1af77ce1375278b097816a8be
2015-07-16 20:24:42 +00:00
dan 8694d6049f Fix compiler warnings in fts5 code.
FossilOrigin-Name: e9bf275cd969eca6fb41384d3637528d6a19f819
2015-07-16 20:17:57 +00:00
dan e3229c19cb Use a WITHOUT ROWID table to index fts5 btree leaves. This is faster to query and only slightly larger than storing btree nodes within an intkey table.
FossilOrigin-Name: 862418e3506d4b7cca9c44d58c2eb9dc915d75c9
2015-07-15 19:46:02 +00:00
drh a8dbd52abb Amplify the comment on renderLogMsg() that explains the problems associated
with calling sqlite3_log() from deep within the memory allocator.

FossilOrigin-Name: a73d7128fbca8dde5e90bd46ee915e39ae07dd1f
2015-07-14 22:43:37 +00:00
mistachkin 532f179cab Fix some harmless compiler warnings.
FossilOrigin-Name: 3de030c904d125ccf41fa1929646b8a002b5341b
2015-07-14 17:18:05 +00:00
dan 3eb9024d68 Fix inconsistencies in formatting of fts5 docs.
FossilOrigin-Name: 5fb4c77163be8d5ff1e2934dda3cc2ad0a6267e8
2015-07-10 17:55:42 +00:00
dan d2454f47e7 Improve the performance of docid merges in fts5.
FossilOrigin-Name: b2de77a01cc5edcea2f98f7916e64cb33e6bd414
2015-07-09 20:46:35 +00:00
dan 3dbd2397f4 Reduce the number of calls to malloc() made by fts5.
FossilOrigin-Name: 898618ccf61d3ad166d9fc742e132d518338b5c3
2015-07-09 19:02:19 +00:00
dan a90579db99 Fix two problems that could cause fts3 auxiliary functions to occasionally misbehave if used with match expressions that contain both OR and NEAR.
FossilOrigin-Name: 372c1db2475f367d54270d5801aff0503745bff4
2015-07-08 17:59:08 +00:00
dan bcf2163771 Merge the Fts5MultiSegIter and Fts5IndexIter objects.
FossilOrigin-Name: 6ace73541ae48aeb19e0c2d9f6772d032d0a7eae
2015-07-07 19:35:11 +00:00
dan 01233d3e54 Add a test case to verify that "PRAGMA data_version" works as expected when an OTA client writes to the database.
FossilOrigin-Name: 6441f2dc9e7f53de64bc15b1d766c885b2635112
2015-07-07 19:07:30 +00:00
dan b2fe306a2e Further optimizations for fts5 b-tree seeks.
FossilOrigin-Name: f37899686c032145f431f81c1072738536c61c7e
2015-07-07 08:29:32 +00:00
dan 656773b675 Speed up seek operations on fts5 b-tree structures.
FossilOrigin-Name: 7b7da1eb435d321fc4283f6aa2161fa1e16f2cf3
2015-07-06 20:27:19 +00:00
dan 30ae503167 Optimize seek operations on fts5 b-trees.
FossilOrigin-Name: 8cf02090ce53ec150492d77d9e5e5f27665bd34f
2015-07-04 18:44:07 +00:00
dan 70c8885417 Fix a problem in the fts5 integrity-check.
FossilOrigin-Name: 4e473559c9bdcf4d0b823e995a3642cd50ebd337
2015-07-04 15:35:02 +00:00
dan 28a10c3c1f Rework the Fts5MultiSegIter structure a bit to make it more efficient.
FossilOrigin-Name: 0778825d0ec9315c70659fae8d0640b209049dd8
2015-07-03 20:47:18 +00:00
dan 00a1351e55 Speed up eof checks on fts5 cursors.
FossilOrigin-Name: 3df4af5d8c28863783b0bc867abfbe31cc96f1b9
2015-07-03 19:13:56 +00:00
dan ec7c1de354 Avoid freeing the array of instance matches each time the fts5 xNext() method is called.
FossilOrigin-Name: b9b0c1e50d77f5d6e02f43fbb100c722cb692cc5
2015-07-03 17:14:18 +00:00
dan a153bbc4c2 Copy fts5.h into the build directory as part of building fts5.c. Also remove code to edit fts5parse.c from various Makefiles and add it to mkfts5c.tcl.
FossilOrigin-Name: 4d8f03dc7b6529cf8d46fc0fcc82112c0394dc13
2015-07-02 20:12:34 +00:00
dan 6544b8643f Minor change to mkfts5c.tcl so that fts5.c can be used according to the instructions in loadext.html.
FossilOrigin-Name: edbcccd349090efff3f975e27451608136a429fc
2015-07-02 18:52:16 +00:00
dan 3abdc44e7b Add fts5_source_id() SQL function, similar to sqlite_source_id().
FossilOrigin-Name: ec2573487cb86664f7f056168a764c28ca8236fc
2015-07-02 18:16:02 +00:00
dan 3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
2015-07-02 15:52:21 +00:00
dan ef648698c8 Fix minor problems in the ota demo application.
FossilOrigin-Name: 6aaaec6e63cdf713b0d67e24a892088ff251c82a
2015-06-29 15:41:50 +00:00
dan 018a93d216 Remove unnecessary lines from fts3.h and main.mk.
FossilOrigin-Name: c1b268ced38259706674ed2fcbf842d3fbd9fb8c
2015-06-26 20:13:56 +00:00
dan 6394d99a0e Fix a segfault that could follow an OOM error in fts5.
FossilOrigin-Name: 713239b8cf2900e8f7d97646c7f350248b4e804f
2015-06-26 20:08:25 +00:00
dan 6fbeb22596 Fix some cases in the fts5 code where a corrupt database could cause a buffer overread.
FossilOrigin-Name: 360c57bbb8bad6cc84af140f61e0282fa06462b4
2015-06-26 18:50:29 +00:00
mistachkin a468027c8d Merge upstream changes. Get the test fixture working with FTS5. Fix compiler warnings.
FossilOrigin-Name: bfcd16089cf8c65130c011a9718bf7812685c841
2015-06-26 16:55:55 +00:00
dan eb37567fe2 Remove fts5 initialization code from core file main.c.
FossilOrigin-Name: c91a93b343b4d3b1a19afdb5d23d1c508f9779d2
2015-06-26 16:42:17 +00:00
mistachkin ed52f9ff48 Initial changes to get FTS5 working with MSVC.
FossilOrigin-Name: ef2052f81e33ca98e85a60f8a78cdd19a7c1c35c
2015-06-26 04:34:36 +00:00
dan 3e65f89ef6 Fix Makefile.in to account for recent changes. Update fts5 test scripts so that they do not run if SQLITE_ENABLE_FTS5 is not defined.
FossilOrigin-Name: 3175220747f5d4dadd17e09fb1899dc782c90926
2015-06-25 20:36:36 +00:00
dan f32fa3116b Merge latest trunk changes into this branch.
FossilOrigin-Name: fcd8f7ce601729dc51d880d16b97040c1be16aa2
2015-06-25 20:16:23 +00:00
dan ed2589f555 Add a script to combine all fts5 code into a single file - fts5.c - that can be used to build an SQLite loadable extension.
FossilOrigin-Name: 46e86b0637248fb4d623c97778cc041eabe3636c
2015-06-25 20:10:24 +00:00
dan 88d702e6bd Add "ON CONFLICT" handling to the spellfix module.
FossilOrigin-Name: 1d04def785b6031de68b7f199d400cbb5c76caea
2015-06-24 17:21:52 +00:00
dan 51ef0f57c7 Improve test coverage of fts5.
FossilOrigin-Name: df5ccea80e8f0da83af5e595b539687006085120
2015-06-23 18:47:55 +00:00
dan c94a08100b Merge latest trunk changes with this branch. Add tests for columnsize=0.
FossilOrigin-Name: ef44c71a22518727030dd90c0139af8973b05841
2015-06-23 15:06:13 +00:00
drh f5ed7ad63e Fix harmless compiler warnings.
FossilOrigin-Name: b0badb99023e23bef0e2064cec58f2b279af0c0b
2015-06-15 14:43:25 +00:00
dan 1153e60cf0 Fix the fts5 xRename() method.
FossilOrigin-Name: 0f7fd51325875fbf0f1eaca3bbbd170ef99c4208
2015-06-10 10:45:34 +00:00
dan bcc2f04c68 Add the "columnsize=" option to fts5, similar to fts4's "matchinfo=fts3".
FossilOrigin-Name: aa12f9d9b79c2f523fd6b00e47bcb66dba09ce0c
2015-06-09 20:58:39 +00:00
dan 4f20cd402b Avoid passing constraints that are unusable due to LEFT or CROSS joins to virtual table xBestIndex() methods.
FossilOrigin-Name: 80ee56dda7db3860f8be5f6968c8745138f8453f
2015-06-08 18:05:54 +00:00
dan 90502c3dc7 Fix a comment in fts5.h.
FossilOrigin-Name: e964b5877497b16cf985d3d847e82529bb3fa4a3
2015-06-06 19:23:32 +00:00
dan 7193b69f52 Fix handling of fts5 rowid constraints in the absence of a MATCH clause. Add tests to cover recently added branches.
FossilOrigin-Name: 3a9cb648138a059862fb438c0787fdd888f5e88e
2015-06-06 16:28:29 +00:00
dan e4449454c5 Make use of range constraints on the rowid field of an fts5 table in full-text queries.
FossilOrigin-Name: 32cbc0ed3699cc21302f0b6a159493117ad4bd4f
2015-06-05 19:05:57 +00:00
dan 6f277eb6bc Fix an fts5 problem in extracting columns from position lists containing large varints.
FossilOrigin-Name: 4ea015ab983300d420ef104cca550b22a6395866
2015-06-03 11:23:30 +00:00
dan f67bb4a21b Change the fts5 multi-column syntax to use parenthesis instead of square brackets.
FossilOrigin-Name: ab85a6fc4f7580278fc9d1f0090fdcf0a90d065b
2015-06-02 19:38:15 +00:00
dan 167fd757f0 Merge latest trunk changes with this branch.
FossilOrigin-Name: c9ffda4abb4390bbc5719e269196e2807b254f97
2015-06-02 18:07:58 +00:00
dan dbef6e752e Reimplement [ec69e09a] so that each call to the xNext() method does not involve two iterations of the match expression tree (only one).
FossilOrigin-Name: 80fe305b3eefb17310a9d6185d1c8cd73ee38b1e
2015-06-02 17:57:01 +00:00
dan 5350d91ed3 Improve performance of the fts5 AND operator.
FossilOrigin-Name: b43e9a5b7a0483ccb102316a4dbc5e32b5bc69ec
2015-06-01 19:17:06 +00:00
dan 34efc82eed Change fts5 expression processing to avoid linear scans of long doclists caused by phrases that match specific columns only.
FossilOrigin-Name: ec69e09a55b4daf1c40aeaaf9ee95091fe86f5c0
2015-06-01 09:15:20 +00:00
dan 3f3074e0c1 Remove the "#include sqlite3Int.h" from fts5Int.h.
FossilOrigin-Name: e008c3c8e29c843ec945ddad54b9688bbf2bdb44
2015-05-30 11:49:58 +00:00
dan a8c024905c Add extra tests and fixes for multi-column matches.
FossilOrigin-Name: ae6794ffa23ef6191bd8834422abf322d978c11b
2015-05-29 19:00:22 +00:00
dan 6d21f42db2 Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>".
FossilOrigin-Name: 0fc0ea20920615f3e48ea2dbe2b7dcd979b0993e
2015-05-29 15:55:30 +00:00
dan e96efb6426 Optimizations for fts5 queries that match against a specific column.
FossilOrigin-Name: b29ac50af0491a780a5a4c0985d88d0e5e014ba3
2015-05-28 19:57:12 +00:00
dan af5cd564f0 Remove some dead code from fts5. Add auxiliary function api tests to the same.
FossilOrigin-Name: 0f9df202cc58097afddb8dad662b7c7fdc2c7d0c
2015-05-28 14:37:26 +00:00
dan 7f64db1343 Simplifications and minor optimizations to fts5 prefix queries that cannot use a prefix index.
FossilOrigin-Name: aef89d9f6aa536efee347367558cf5d4ea81b238
2015-05-26 18:22:01 +00:00
dan add0804943 Avoid using a zero-sized array within a struct in fts3 code.
FossilOrigin-Name: b05cae36cedd98d59813e637f328a52eee7ef0d2
2015-05-26 17:29:48 +00:00
dan 494016a424 Avoid redundant loads from the %_data table in the fts5 code.
FossilOrigin-Name: 02069782f8b7896a582582c79185b50418622736
2015-05-25 11:46:33 +00:00
dan 3fb37f097c Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions.
FossilOrigin-Name: f1e942a1dda496a509741e9cc2a17e8b4dac63a3
2015-05-25 10:57:13 +00:00
dan 2e7d35e2fe Avoid making redundant copies of position-lists within the fts5 code.
FossilOrigin-Name: 5165de548b84825cb000d33e5d3de12b0ef112c0
2015-05-23 15:43:05 +00:00
drh 18070e08c5 Fix incorrect validation of the size of the BLOB returned by a geometry
function on the RHS of a MATCH operator in RTree.  The old code worked for
64-bit systems (by chance) but fails on 32-bit.

FossilOrigin-Name: 10cc44f5a63e6cadf6940bb7310a177ee7fa6ee2
2015-05-22 23:17:28 +00:00
drh fd29973f11 Update with fixes and enhancements from trunk.
FossilOrigin-Name: 9605d008f5c763137e9734d033fe4510ca316858
2015-05-22 18:40:48 +00:00
dan 6b76418e67 Have r-tree prefer to use the constraint "col MATCH ?" over "rowid = ?".
FossilOrigin-Name: b9fb95194d4f7c535f1d175fd2e18d69d76e8fc4
2015-05-22 16:08:42 +00:00
dan 0d45967b7f Increase test coverage of fts5_vocab.c.
FossilOrigin-Name: 065ab83a6ce36e16d3b95a61505aa3cff0bfea84
2015-05-22 07:44:44 +00:00
dan 21b7d2a9b8 Improve test coverage of fts5_unicode2.c.
FossilOrigin-Name: fea8a4db9d8c7b9a946017a0dc984cbca6ce240e
2015-05-22 06:08:25 +00:00
drh 4f03f413bc Add the sqlite3_value_dup() and sqlite3_value_free() interfaces. Use
these interfaces to enhance R-Tree to add the
sqlite3_rtree_query_info.apSqlParam field.

FossilOrigin-Name: a7ee40c4fc62843ac5b96ba47ca14a66e8cd6961
2015-05-20 21:28:32 +00:00
dan 8c1f46de50 Improve test coverage of fts5_tokenize.c.
FossilOrigin-Name: 0e91a6a520f040b8902da6a1a4d9107dc66c0ea3
2015-05-20 09:27:51 +00:00
drh 7ac07bda73 Fix the MSVC makefile so that it works with OTA. Fix a harmless compiler
warning in FTS3.

FossilOrigin-Name: c7b16645307d6e46c4153693d6806269de64955f
2015-05-19 22:56:45 +00:00
drh 856d446efe Allow R-Tree geometry functions to take 8-byte BLOB arguments which are
passed directly through to the underlying callback, and which can be used
to pass pointers into the callback.

FossilOrigin-Name: b271ed56532a78323accc8a7cb348d55f95c350e
2015-05-19 22:20:48 +00:00
dan 116eaee4a0 Add tests for fts5 tokenizers.
FossilOrigin-Name: 4f90ba20e2be6ec5755fe894938ac97342d6fbf6
2015-05-19 19:37:09 +00:00
dan 815bd625c2 Remove the sqlite3ota_open_v2() API. Add a new parameter to sqlite3ota_open() instead.
FossilOrigin-Name: c74e0bc481ce995f83ca8384e05dfbe068a0ae85
2015-05-19 17:48:11 +00:00
dan a6fb464878 Allow OTA update state data to be stored in a database separate from the OTA update database.
FossilOrigin-Name: 5af8db56af457d60ea030d84666ca7fffb6821fe
2015-05-19 16:22:58 +00:00
dan b10210ea1b Fix a memory leak that could follow an OOM condition in fts5.
FossilOrigin-Name: de9f8ef6ebf036df5a558cd78fb4927da2d83ce8
2015-05-19 11:32:01 +00:00
dan 09963b27b8 Improve test coverage of fts5_storage.c.
FossilOrigin-Name: 4dfe2e5871704125338620705e6638f36baaacd7
2015-05-18 20:34:59 +00:00
dan 23390b1b2b Fix a bug in test script fts5fault4.test.
FossilOrigin-Name: a508059305641c2cb53197b0f6e93d7b723697e2
2015-05-18 18:24:41 +00:00
dan 5211e54f95 Add a test for an untested branch in fts5_expr.c.
FossilOrigin-Name: ce08206b5c5bf6a81decf20c99a524dd92c3715b
2015-05-18 18:03:06 +00:00
dan 27aac274b9 Improve test coverage of fts5_config.c.
FossilOrigin-Name: 47dbfadb994814c9349d4c9c113b862c2e97c01a
2015-05-18 17:50:17 +00:00
dan e0569a4aae Further test coverage improvements for fts5.
FossilOrigin-Name: 927d9a64e13c6b768f0a15475713192fcfaaf9e7
2015-05-16 20:04:43 +00:00
dan 369e4129b9 Improve test coverage of fts5_index.c.
FossilOrigin-Name: 7aea8c6d99737c6c72078e0b4b9c5f8186021aa0
2015-05-15 18:13:14 +00:00