Commit Graph

1253 Commits

Author SHA1 Message Date
dan
3c060c2aef Fix a problem in the fts5txt2db.tcl script causing it to fail when creating tables with more than 8 columns.
FossilOrigin-Name: 68ee426a6432ac63655909ff70cb22bde33cf28c
2015-10-19 15:59:08 +00:00
dan
0206920e27 Add tests to cover new branches added to the fts5 code by the previous commit.
FossilOrigin-Name: cf618334a8a05a067e844236e4d461b2e5706da6
2015-10-17 20:34:49 +00:00
dan
05903e6a87 Handle equality and range constraints on the "term" column of fts5vocab tables. Also have the same module report columns using names instead of indexes.
FossilOrigin-Name: 6020d96ab47a0bebe41f691a3f44fd55d8b2d2b7
2015-10-17 19:49:01 +00:00
drh
49472655d4 Fix the JSON1 extension so that it does not depend on isdigit() and
strtod() from the standard library when compiled into SQLite as part of the
amalgamation.

FossilOrigin-Name: bc9a9a60c31ebf9b11ac89ae5f99a3b66d6efc67
2015-10-16 15:35:39 +00:00
drh
b9e8f59b63 Form-feed is not valid whitespace for json.
Fix for ticket [57eec374ae1d0a1d]

FossilOrigin-Name: 28957d635961c525f735a52b8ffe3e69ccf31382
2015-10-16 15:16:06 +00:00
dan
5f93ccd7fc Tests and minor fixes to improve coverage of FTS5 code.
FossilOrigin-Name: f4de6d450e143cb1dd5447800d4b178e3db114c8
2015-10-14 21:08:48 +00:00
mistachkin
cdabd7bd50 Fix harmless compiler warnings.
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
2015-10-14 20:34:57 +00:00
dan
8d32e8042c Avoid exporting sqlite3_json_init() from amalgamation builds.
FossilOrigin-Name: 8463f7e7908d0a96ed036377dfa3805328c62925
2015-10-14 18:45:42 +00:00
dan
f4f968443b Add the missing 'extern "C" {...}' blocks to fts5.h and sqlite3rbu.h.
FossilOrigin-Name: 47a2ce97d585e1760ffcf760c0de1753677d5713
2015-10-14 10:56:34 +00:00
drh
cec7dec006 Remove the unused fts5ExprColsetTest() routine.
FossilOrigin-Name: 9ecafc0c94102dca6da192f400278399464086fb
2015-10-12 22:31:02 +00:00
mistachkin
f2c26ed17b Fix a couple harmless compiler warnings.
FossilOrigin-Name: 7f896a971c5953d5370215ecd834d1fb711b4263
2015-10-12 22:20:29 +00:00
dan
df705d8185 Further tweaks to improve fts5 prefix query performance.
FossilOrigin-Name: 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
2015-10-12 19:12:29 +00:00
drh
72724da95a Remove another instance of strcpy() from FTS5, to mollify OpenBSD.
FossilOrigin-Name: 35e6248abb4435a8b26d270092b856beff867406
2015-10-10 15:57:20 +00:00
drh
c5971cc859 Omit the use of strcpy() in FTS5 since OpenBSD hates strcpy().
FossilOrigin-Name: bc24a5bbfd95df3518611b221de69b73776111bc
2015-10-10 15:11:49 +00:00
drh
9a4718ff73 Add some #ifdef-ery to json1.c to avoid a duplicate typedef when used in the
amalgamation, since some compilers become upset over duplicate typedefs.

FossilOrigin-Name: de28acd42f29693341feb884b7223cd3d2b96730
2015-10-10 14:00:37 +00:00
drh
a0882fad7c The previous fix to JSON1 was not complete. A few more tweaks are needed
for correct handling of all oversized integers.

FossilOrigin-Name: 4a47f0177851721cdf6c981df962b6e8862372e7
2015-10-09 20:40:44 +00:00
drh
8deb4b8b17 Fix the JSON1 extension so that it renders integers outside the range
of -9223372036854775808 to +9223372036854775807 as floating-point numbers.

FossilOrigin-Name: ae736e35fb59c9aed33a8c805cf2ecdee528051f
2015-10-09 18:21:43 +00:00
drh
b5c8d27c7d Add the JSON1 and FTS5 extensions to the amalgamation. Add new options
to ./configure: --enable-json1 and --enable-fts5.

FossilOrigin-Name: 1eb769933108cf401b25c8d784ea23f27324a0bd
2015-10-09 13:39:51 +00:00
dan
0008ce7cc5 Makefile and fts5 changes so that fts5 works with main.mk. Still does not work with Makefile.in.
FossilOrigin-Name: 6ebac32d8378e46f44a6eb55aaa30baf4d7dde7d
2015-10-09 11:09:47 +00:00
drh
45fac88951 Trying to get FTS5 tests to build and run using main.mk. Not yet working.
FossilOrigin-Name: 206174f9b154c590a3e2e94236db38e111d64bdb
2015-10-09 01:42:49 +00:00
dan
d3b90a2e41 Remove calls to "load_static_extension" from fts5 test scripts.
FossilOrigin-Name: 53dd7c4c65842c5c9ed3e84edd62e24f2c9be15b
2015-10-08 21:02:29 +00:00
dan
23fda8242c Further tweaks to main.mk so that fts5 builds as part of testfixture.
FossilOrigin-Name: 97e11b79a52fb1c57f0afd2dbd6ee66cf00a922d
2015-10-08 21:00:46 +00:00
dan
56eac26897 Fix further fts5 compilation issues.
FossilOrigin-Name: 0d7f301fb03c8546a5dbce5d061a1d74e3c86bcf
2015-10-08 20:40:18 +00:00
dan
351ccb3825 Fix two more warnings in fts5 code.
FossilOrigin-Name: 5343e601fb6f0f24f4e4d5d442f7b5b074734143
2015-10-08 20:28:33 +00:00
dan
056d7acf25 Fix a couple of errors in fts5 compilation. This branch still does not build with -Wall -Werror.
FossilOrigin-Name: e9e76e12812b9b42b1142246b71cea98e26903aa
2015-10-08 20:11:42 +00:00
drh
50065656f7 First attempt to add json1 and fts5 to the amalgamation. This check-in does
not compile.

FossilOrigin-Name: d820a1bd1b8dd702d1b3a099af393530cc5a5d51
2015-10-08 19:29:18 +00:00
mistachkin
b2682eb14b In the RBU extension, use MoveFile() instead of rename() on Windows CE.
FossilOrigin-Name: df9ef61f1ba2e3a3420fae84e3eaa2e0e235bc28
2015-10-08 17:35:51 +00:00
dan
98e713bb8d Further optimizations for fts5 prefix queries without a prefix index.
FossilOrigin-Name: 83dc1ff7fa010715ca7f406a572f4ee444a967d7
2015-10-07 19:06:21 +00:00
mistachkin
08114efde4 Fix harmless compiler warning in FTS5.
FossilOrigin-Name: 13adcd038fc20dd1b6f344f79b449b4034f8f8f2
2015-10-07 17:06:17 +00:00
dan
22c7cca21f Allow the subsitute character (codepoint 26 - 0x1A) to appear in fts5 barewords.
FossilOrigin-Name: baec1b96cb64a6a6317143a8df841e502afe3914
2015-10-07 17:01:22 +00:00
dan
d46cc33131 Add tests for fts5 phrase queries with column filters.
FossilOrigin-Name: f20f9f813f00cefcd427e506a09b0b973c782e3f
2015-10-07 13:24:27 +00:00
dan
92099b0753 Add further tests for fts5 prefix queries.
FossilOrigin-Name: accdc98b1291f07b802fd23f3ebc7dbc02ba09d3
2015-10-07 09:02:50 +00:00
dan
a2507137f3 Optimizations for fts5 expressions that filter on column. More still to come.
FossilOrigin-Name: bf1607ac155018573ca40fb58aca62c5fea7e60b
2015-10-06 20:53:26 +00:00
dan
6f832d3dff Improve performance of prefix queries without a prefix index on fts5 tables.
FossilOrigin-Name: f2f0184e9e1c9f121ee2ac864cd28c8cd8efecb5
2015-10-05 19:41:16 +00:00
dan
f24bebe37a Update fts3 so that expressions to the left and right of a NOT operator are balanced. This prevents relatively small expressions (a dozen terms or so) that are children of NOT operators from triggering the "expression tree is too large" error.
FossilOrigin-Name: d6b66cd7b89fbd964f798d160a34caac0ba7347a
2015-10-05 15:39:45 +00:00
dan
2c1023df09 Add fts5txt2db.tcl, a tool for creating sample fts4/5 databases from text files.
FossilOrigin-Name: 44f1ce30d1e446c9ee5f8bd8a62119e00356aa0e
2015-10-05 11:57:09 +00:00
dan
a0f197b842 Update fts5 to support the table function syntax. "... FROM fts5_tbl WHERE fts5_tbl MATCH ?1 AND rank MATCH ?1" can now be written "FROM fts5_tbl(?1, ?2)".
FossilOrigin-Name: 41d17d9e24303aed6d549ea23c62b93bf2f71691
2015-10-03 15:38:57 +00:00
dan
06f8c635ef Add tests for the rtree module to verify that attempts to insert non-integer primary key values or non-numeric dimensions into an rtree table are handled correctly.
FossilOrigin-Name: f653fce90846b700441e8fa5f1930c1ec5e38e31
2015-10-03 12:23:18 +00:00
dan
bca189d5d8 Update fts5 to avoid using a statement journal for UPDATE and DELETE operations that affect at most a single row.
FossilOrigin-Name: 5c83b9db46d61cfa76a1abed50467e2f02db0eb0
2015-10-02 20:04:30 +00:00
dan
5171e911f2 Fix an fts3 bug causing NEAR queries on uncommitted data to malfunction.
FossilOrigin-Name: 6f90839e91024e2006042f5eb7f21ca5b47a9b4a
2015-10-01 18:31:29 +00:00
dan
e60856980a Add the sqlite3_index_info.idxFlags field, allowing xBestIndex() implementations to specify to SQLite that a strategy may visit at most one row. Add support for this to fts3/4. Omit the statement journal from virtual table UPDATE and DELETE operations that are guaranteed not to affect more than one row.
FossilOrigin-Name: a1d08fd3d0419da8c22355d48c6d83eed6fd7e07
2015-09-29 16:47:53 +00:00
dan
4ad0971919 Fix an off-by-one error in test function fts5_decode().
FossilOrigin-Name: 3a9f076250d9559d8ea94ba44095ba3ddbc5542d
2015-09-29 12:19:51 +00:00
dan
b3deb4ea64 Change the name of the new sqlite3_index_info.flags field to "idxFlags". Add documentation for the same to sqlite.h.in.
FossilOrigin-Name: f61203bc0fb0b62e12d6270e8d9063e445a5a252
2015-09-29 11:57:20 +00:00
dan
0f40037eee Also allow UPDATE on virtual tables to use the onepass strategy.
FossilOrigin-Name: 1aa27d706db9b2e21737ce4b94b47ecd12c2570f
2015-09-28 20:03:49 +00:00
dan
d943b27596 Update fts3 to use the onepass strategy for delete operations.
FossilOrigin-Name: fffab4f70f85eeb2acbb89534064a6e397c39384
2015-09-28 15:23:29 +00:00
drh
2f20e13b83 Enable adding JSON1 by appending the json1.c source file to the amalgamation
and compiling with -DSQLITE_ENABLE_JSON1

FossilOrigin-Name: 33404b2029120d4aabe1e25d484871810777e934
2015-09-26 17:44:59 +00:00
drh
8cb15cc5ae Another (smaller) performance optimization for the JSON parser.
FossilOrigin-Name: c43daa8c78df99f62dd4d3c83708a3a8eff92496
2015-09-24 01:40:45 +00:00
drh
9567794fd9 Performance optimizations on the JSON parser.
FossilOrigin-Name: 7dd4b07a42eb84589d34430b9d7bfa88fbd743eb
2015-09-24 01:06:37 +00:00
drh
f27cd1f3aa Do not allow a comma at the end of a JSON array or object.
FossilOrigin-Name: 7c7a3f3e9bc2f7a7334d8d6eae183d83f22097be
2015-09-23 01:10:29 +00:00
drh
3d1d2a97a5 Futher simplifications to json1.c. Also an obscure bug-fix in the initial
output of json_tree() when using a path to an object contained within an array.

FossilOrigin-Name: fcb1e327a64134e5ac539ec3d52733422d3061fd
2015-09-22 01:15:49 +00:00
drh
8cb0c83cce Fix json_set() so that it can overwrite a value that was previously overwritten
during the same call.

FossilOrigin-Name: 0f16041647993975c316203c7e11f06e27640136
2015-09-22 00:21:03 +00:00
drh
a8f39a9b5d Simplifications to the json1.c logic.
FossilOrigin-Name: 1646a2bd85f87d03a14fcaec288107f5f6411c6f
2015-09-21 22:53:16 +00:00
drh
dc38495715 Fix a memory leak in json1.c that could occur after misuse of json_object().
FossilOrigin-Name: 394b81b11f261a9b179a034bc2a2d2dff0765e3b
2015-09-19 18:54:39 +00:00
drh
4977ccf143 Fix an off-by-one error (really off-by-2 in this case) in the buffer
resize logic of json1.

FossilOrigin-Name: d2a027372a5a6efc0f9b6f605093d865ae1c6788
2015-09-19 11:57:26 +00:00
dan
2e8f5517f7 Avoid passing (signed char) values directly to isspace(), isalnum() or isdigit() in json1.c. Cast the value to (unsigned char) first.
FossilOrigin-Name: 6713e35b8a8c997aa2717e86ce6dcd63bb993477
2015-09-17 17:21:09 +00:00
dan
cf346d7a9c Fix a crash in FTS5 caused by specifying NULL as the rhs of a MATCH operator.
FossilOrigin-Name: e8ed62f82e8acc40b818bf86fafe3d480687514e
2015-09-17 09:48:01 +00:00
dan
ec9e8f004e Fix typo in fts5.h.
FossilOrigin-Name: 07721c6c24f651d8e281c29227b113d292f6b0de
2015-09-16 12:31:39 +00:00
dan
6609bab24d Add test cases to cover fts5 integrity-check code.
FossilOrigin-Name: 1d018c35b9e81982df036f5e62a4a42219b54e02
2015-09-15 14:39:17 +00:00
dan
128ea696ea Fix a problem with fts5 "ORDER BY rowid DESC" queries and large terms.
FossilOrigin-Name: b26d8f79c62cb71b4c3ea5df9d0e09558bbbd794
2015-09-15 11:58:33 +00:00
dan
dafeda754b Improve test coverage of fts5_index.c.
FossilOrigin-Name: c1f76686cee3918b1be785a4071d68cb3afda0ef
2015-09-14 19:51:05 +00:00
mistachkin
16a93128f8 Fix harmless compiler warnings.
FossilOrigin-Name: bfc7b84b766860d2e410702ba7c1166d7328309a
2015-09-11 18:05:01 +00:00
dan
bcb04b6128 Fix some compiler warnings in fts5 code.
FossilOrigin-Name: 0dc436116e55e3fd55eb6085ada71e099069b32d
2015-09-11 14:15:46 +00:00
drh
f5ddb9c214 Take out the goofy '$$' path syntax. Instead, use subtypes to communicate when
a string is JSON.  Add the json() function that validates and minifies the
JSON and sets the appropriate subtype.

FossilOrigin-Name: 8a80d6459e246ec1b38325e1cbd1e862157138b3
2015-09-11 00:06:41 +00:00
dan
33c47f565c Modify the fts5 leaf page format to permit faster seek operations. This is a file-format change. Any existing databases can be upgraded by running the fts5 'rebuild' command.
FossilOrigin-Name: 0c0c4ae971e54efc526eed7bd071c90dfadb95ff
2015-09-10 17:23:37 +00:00
drh
383de6918c Create separate "path" and "root" columns in the json_each() and json_tree()
virtual tables.  "Root" is the 2nd parameter and is fixed.  
"Path" varies as json_tree() walks the hierarchy.

FossilOrigin-Name: 127cce3eb96b819005832997e0a082df9fb96f0b
2015-09-10 17:20:57 +00:00
dan
204debf3a3 Increment the fts5 version value to indicate that the on-disk format has changed.
FossilOrigin-Name: 99de5e3613d557728dd196353516bc7cf64a0e6c
2015-09-10 16:39:38 +00:00
dan
56c8634853 Fix a segfault in fts5 that could occur if the database contents were corrupt.
FossilOrigin-Name: 4931e37da4d2c26d7afc5432f7f0d534b51a85fa
2015-09-10 16:19:01 +00:00
dan
40d45f8960 Merge latest changes from trunk. Including fts5_expr.c fixes.
FossilOrigin-Name: 716e7e747714d6af502f6a87ca8d789bb7ce162a
2015-09-10 15:52:42 +00:00
dan
b97efa0ba3 Update description of on-disk format in fts5_index.c.
FossilOrigin-Name: 85aac7b8b6731e2f6880b80cfd62d431ea059799
2015-09-10 15:49:16 +00:00
dan
f679d97d18 Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages.
FossilOrigin-Name: ffe2796ac9244c62325fce4960f26c653321623c
2015-09-10 10:01:30 +00:00
dan
a5aa8e1db2 Change the array of 16-bit offsets at the end of each page to an array of varints.
FossilOrigin-Name: fab245bea4f283714c17bca22428d5eb4db5935a
2015-09-10 05:40:17 +00:00
drh
852944eb9b Fix the json_tree() scan for the case when a path is supplied. Add new
json1 test cases.

FossilOrigin-Name: 6adc7de76acee6cfb5ff761739e7a8de7b5bf4b2
2015-09-10 03:29:11 +00:00
mistachkin
f5eac36a64 Fix harmless compiler warning in FTS5.
FossilOrigin-Name: 86146a731d75eb25279c0e072c0bdda593de905d
2015-09-09 23:54:46 +00:00
mistachkin
4b03efb695 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: 2cdb18778f20baa902e54f218ba26d2bacf0801e
2015-09-09 17:17:22 +00:00
dan
f06bd2c132 Fix a bug in preprocessor macros within fts5_main.c.
FossilOrigin-Name: 0eb2b9521fad6fa36e6fa374c2bc1f70b5180f7c
2015-09-09 08:15:06 +00:00
dan
fe8e2eba0a Remove the 0x00 terminators from the end of doclists stored on disk.
FossilOrigin-Name: 00d990061dec3661b0376bd167082942d5563bfe
2015-09-08 19:55:26 +00:00
dan
60f8139a96 Use macros to make the code in fts5_index.c easier to read.
FossilOrigin-Name: 67ff5ae81357eb7fa28049bb724a22cb6f52e076
2015-09-07 08:14:30 +00:00
dan
e386a1ba25 Experiment with a different fts5 leaf page format that allows faster seeks.
FossilOrigin-Name: a1f4c3b543eed84e808f6b901a38179786fffe16
2015-09-05 19:52:08 +00:00
dan
4dfe98a8c4 Enhance showfts5.tcl so that it can optionally display the number of terms in each segment.
FossilOrigin-Name: d648ddd93de039820f5abe064c7bc1318cd9d6b1
2015-09-04 11:13:00 +00:00
dan
f8efe64649 Rearrange code in fts5_expr.c so that synonym support does not slow down the common case.
FossilOrigin-Name: 801882817f1d895aef1426f9a7196bd140b807c3
2015-09-03 19:48:02 +00:00
dan
cce413202a Merge enhancements from trunk.
FossilOrigin-Name: 4b49fe996989fe42d2bb1c24e7193fef09c5fc50
2015-09-03 18:57:52 +00:00
dan
ea8b93ee20 Add documentation for fts5 synonym support.
FossilOrigin-Name: 58aa1f435959852df74f1bca8e0bdbc4f47c256a
2015-09-03 18:05:09 +00:00
dan
ea623630e2 Add tests to improve coverage of fts5_varint.c.
FossilOrigin-Name: 89f24f31a8f7d7cb0a66ee53523881f566dcb035
2015-09-03 15:37:26 +00:00
dan
e1c28d7229 Remove some more code from fts5_index.c by consolidating similar functions.
FossilOrigin-Name: 59ae30b97b40faa363c55aa2664dead9eaeeddc0
2015-09-03 14:22:27 +00:00
dan
5d144875bc Remove dead code from fts5_index.c.
FossilOrigin-Name: 8a0a9b01e74072ee52fe393311ad591208fbbf7c
2015-09-03 11:17:52 +00:00
dan
9a9e3531f7 Fix the fts5 integrity-check so that it works with columnsize=0 tables.
FossilOrigin-Name: 11b887b15eaee57ea2577c763e70494f1e251275
2015-09-03 11:00:50 +00:00
dan
2a6ecb3141 Fix a memory leak in fts5_expr.c.
FossilOrigin-Name: 399932a181437d84cd57234e76c4b65da1e4ac5f
2015-09-03 10:27:02 +00:00
dan
9c671b741c Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same.
FossilOrigin-Name: bdedd838bb3028c586bcc9f643852ce1364adb49
2015-09-02 19:48:55 +00:00
dan
50ea76e748 Fix an issue with fts5 synonyms and NEAR(...) queries.
FossilOrigin-Name: f2e590700d8eb30da8ac3a92f6ca92a2bbe9c1fe
2015-09-02 18:56:01 +00:00
dan
bea34fc53a Fix a problem with fts5 synonyms and phrase queries. Also fix an OOM handling bug in fts5.
FossilOrigin-Name: a4c35fa2c94fe34b376670244fe72303c99868c1
2015-09-02 17:34:22 +00:00
dan
583611df65 Fix a problem handling OOM conditions within fts5 queries that feature synonyms.
FossilOrigin-Name: 11fa980897c6c7be218bbd9c4cd8253272d2c300
2015-09-02 14:17:38 +00:00
dan
df998c3d4f Fix a problem with fts5 synonyms and the xQueryPhrase() auxiliary function API.
FossilOrigin-Name: cf3e45e76d23e10ee06296c3561a341591597a04
2015-09-02 08:22:41 +00:00
dan
d917ad019c Fix a problem that occurs when more than 4 synonyms for a term appear within a single row.
FossilOrigin-Name: cd359550bdc2bf7be4c294b60130c9fc3f583902
2015-09-01 18:44:33 +00:00
dan
3faa07ee04 Add tests for fts5 synonyms implemented by adding extra terms to queries. And fixes for the same.
FossilOrigin-Name: dbcb73802b88f76be17f09f3eb83ffac542de633
2015-09-01 18:08:36 +00:00
dan
cb8b58caf7 Begin changes to allow synonym support by adding multiple terms to a query (an alternative to adding multiple terms to the FTS index).
FossilOrigin-Name: ad7feaed4cd6b1d6e6376bb82d1f5664ddd083f3
2015-08-31 20:06:06 +00:00
drh
3ad93bba84 Enhance the json_extract() function so that if given multiple PATH arguments
it will return a JSON array with all of the answers.  Also update comments
within the json1 extension to reflect stricter interpretation of JSON and PATH
arguments.

FossilOrigin-Name: 1da60c3dda4254620052a83c853c2d2b6dd5009f
2015-08-29 19:41:45 +00:00
dan
0cb01f5c2c Add a test for an fts5 tokenizer that supports synonyms by adding multiple entries to the fts index.
FossilOrigin-Name: 98d07d16cab92f1e7001afbe370df3ec6343fc1f
2015-08-29 18:46:12 +00:00
drh
d29759288e Fix the build with -DSQLITE_OMIT_VIRTUALTABLE.
FossilOrigin-Name: 752918def7231f7846b3e985c9953a1cc825ab6b
2015-08-29 17:22:33 +00:00
drh
d1f0068d19 Do not consider an empty string to be valid JSON. Add some additional
JSON test cases.

FossilOrigin-Name: fd19ff029f128f478f69910352a6f8b84262ce1d
2015-08-29 16:02:37 +00:00
dan
ee0c0a8de3 Another change to the fts5 tokenizer API.
FossilOrigin-Name: fc71868496f45f9c7a79ed2bf2d164a7c4718ce1
2015-08-29 15:44:27 +00:00