Commit Graph

54 Commits

Author SHA1 Message Date
dan
fb8e71c584 Fix a case in the fts3 offsets() function where a corrupt database record could lead to dereferencing an uninitialized pointer.
FossilOrigin-Name: 7b7d31a6153b1505288eb3e849d0d9ef9e88e961c7b2f918ef5582fd77990f6d
2021-10-20 11:40:34 +00:00
drh
4bb4293b2a Limit parenthesis nesting depth in FTS3 expressions to
SQLITE_MAX_EXPR_DEPTH.

FossilOrigin-Name: 5ddd8032ef9ecd5b53909d304c8f1375a72f270fd7810964cb4e385e25bedd4f
2021-03-06 15:13:26 +00:00
dan
fc7f31742d Fix an integer overflow in fts3 causing a usan error.
FossilOrigin-Name: e256f85289a78e629acdf83e5bf1f8df2a0ffb3d559738eb9e49db6c228dc8c0
2020-04-22 11:11:17 +00:00
dan
68c1f9ce9a Have fts3 ignore empty sets of parenthesis if built with
SQLITE_ENABLE_FTS3_PARENTHESIS.

FossilOrigin-Name: c93c6b45a317c40eb5c0abb6620d21f5821a601632c791e11e5ce62e039eccda
2019-01-14 11:56:13 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
2019-01-08 20:02:48 +00:00
drh
6bfd973ce6 Fix compiler warnings in FTS3.
FossilOrigin-Name: de508e831a43f02c3b354a24ea34798da03c163bae5521c852c9ddd252723739
2018-04-28 04:10:55 +00:00
dan
03db962ab5 Update test script fts3expr4.test so that it always creates fts3 tokenizers in
the "en_US" locality.

FossilOrigin-Name: 576a8f69ae25883f752e58953624e9f7126db998bebaa1f07f7c2ec47aaecabe
2018-04-27 18:05:01 +00:00
dan
f24bebe37a Update fts3 so that expressions to the left and right of a NOT operator are balanced. This prevents relatively small expressions (a dozen terms or so) that are children of NOT operators from triggering the "expression tree is too large" error.
FossilOrigin-Name: d6b66cd7b89fbd964f798d160a34caac0ba7347a
2015-10-05 15:39:45 +00:00
drh
abf582509d Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3,
and to ensure that error messages are not overwritten and thus leaked.

FossilOrigin-Name: 605347e087ec5eb817f3c94f4616abe26c7ab66b
2015-05-01 14:07:30 +00:00
dan
6e1a037357 Allow FTS tokenizers to choose whether or not to consider the "*" character part of tokens or not. This restores the pre-[e21bf7a2ad] behaviour. Also fix a problem causing FTS to interpret tokens beginning with "*" characters as EOF.
FossilOrigin-Name: 49dfee7cd1c9ab2901b8a871a6cd00b2ead76801
2014-10-09 15:08:17 +00:00
drh
b7231a8cf0 Fix a harmless compiler warning in the FTS4 logic.
FossilOrigin-Name: 64a869ad2388d9d62601b93d5139f2dc57f260f7
2014-05-27 11:54:48 +00:00
dan
d2d8ca600e Fix the way parenthesis in MATCH expressions are handled by FTS if the tokenizer considers them to be token characters.
FossilOrigin-Name: e21bf7a2ade6373e94ea403c665f78e1ad22143f
2014-05-07 19:59:36 +00:00
dan
d6dd5def4d Fix a bug preventing FTS from correctly processing bracket tokens that are immediately preceded by characters that are neither whitespace or token characters.
FossilOrigin-Name: 49be646cd981f8ff0434cf90d2748afa30260017
2013-11-04 08:56:22 +00:00
dan
5f0b33c2cb Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option.
FossilOrigin-Name: 24fc9d4438a5615dd20af5419456166df83a72ea
2013-06-11 14:22:11 +00:00
drh
acf9c2b555 Add an assert() to fts3_expr.c in order to silence a clang warning.
FossilOrigin-Name: d8dc2c5fb554eb5898ddc8d703c6a360af7fe023
2013-05-06 14:57:48 +00:00
dan
3a01b600d3 Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
dan
181f4f789d Fix an off-by-one in the code for limiting the depth of FTS expression trees.
FossilOrigin-Name: 72ac73189c3577740a77d2ea2fc7118391c0703f
2013-04-29 17:12:06 +00:00
drh
6f77140f78 Fix harmless compiler warnings in the FTS expression parser.
FossilOrigin-Name: 3c78af8c535e16518f18733325f4cd1df7ec8282
2013-04-26 13:14:19 +00:00
dan
aa29c86e49 Rebalance FTS expressions after parsing to limit recursion during evaluation. Avoid recursion when deleting FTS expression trees. Enforce a limit on the depth of an expression tree.
FossilOrigin-Name: f968d43f80cc2f236e7d09ba1e8278343e2b6976
2013-04-25 20:34:02 +00:00
mistachkin
48864df97d Many spelling fixes in comments. No changes to code.
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
2013-03-21 21:20:32 +00:00
dan
6c42aa4a0e Initialize variables used as outputs of custom tokenizer method calls in fts4.
FossilOrigin-Name: 514505a84bcc677518d60a42896598f85ed78265
2012-10-17 20:15:10 +00:00
dan
996073b3ae Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater.
FossilOrigin-Name: f8e9c445dd358c40e5a7bf3756b9f291909dbea7
2012-03-03 18:46:41 +00:00
dan
e459b07601 Merge the fts4-content branch with the trunk.
FossilOrigin-Name: 8a4077057ddeb08e8edc5f20a75abaaba7a278ba
2011-10-19 16:20:40 +00:00
dan
9743948386 Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables.
FossilOrigin-Name: df36ac948179f37b432a88701b6c79299e073ce8
2011-10-19 15:52:48 +00:00
dan
3f1ea8d114 If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match.
FossilOrigin-Name: 63ac33c860eb32ce96699f06bf83121cec2ffaca
2011-10-18 19:39:41 +00:00
drh
086e4913b5 Fix a few minor and harmless clang warnings in FTS3 and RTREE.
FossilOrigin-Name: b3324f6cc27c3bfb32b12eacace2fc731c2dd644
2011-10-14 22:57:03 +00:00
drh
6b96771b82 Changes to #ifdefs so that the build goes correctly if the only FTS macro
defined is SQLITE_ENABLE_FTS4.

FossilOrigin-Name: a0b43a320e6491de7070966ed7c3ec55fd660a85
2011-06-16 00:54:45 +00:00
dan
abf2545ed9 Fix various issues to do with deferred tokens, NEAR expressions and matchinfo().
FossilOrigin-Name: 3972a787df5ec253b99b148385655e7b68d851fa
2011-06-08 18:39:07 +00:00
dan
126ba6c0ac FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too.
FossilOrigin-Name: a4c7e2820824e82580730c36f85aede2efa66754
2011-06-03 18:00:19 +00:00
dan
e414854800 Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now.
FossilOrigin-Name: 28149a7882a1e9dfe4a75ec5b91d176ebe6284e9
2011-06-02 19:57:24 +00:00
dan
382874fc5c Minor changes made while planning a larger change.
FossilOrigin-Name: 84097a4c759b1d65890af885f137d3cb16eef584
2011-05-28 15:57:40 +00:00
dan
0cb3a1eca4 Fix compiler warnings discovered while building SQLite on [http://www.devio.us/].
FossilOrigin-Name: 5602ec95aa2a74d0624bb6c7d53e7a0d35536253
2010-11-29 17:55:18 +00:00
drh
6ea28d6d84 Fix various compiler warnings.
FossilOrigin-Name: c412f61229b6ab1ac90b932afd56f7c5e3ba1cfe
2010-11-26 16:49:59 +00:00
dan
378d0ab97b Fixes for the matchinfo() function related to FTS4 common token handling.
FossilOrigin-Name: deb80eac9112d21835dfd3cee08ed8f09d975bf7
2010-10-23 19:07:30 +00:00
dan
4f7c5e684a Experimental changes to fts4 to try to selectively avoid loading very large doclists.
FossilOrigin-Name: 5ae0ba447a561e3b6637b52f9b83a9fc683d2572
2010-10-19 14:07:59 +00:00
drh
c78db070a3 Remove all vestiges of ctype.h from FTS3.
FossilOrigin-Name: b8b465ed2c56b23b50334680a2a940885b1ac530
2010-08-06 19:00:12 +00:00
dan
a98af17af6 Add experimental implementation of FTS3 functions matchinfo() and snippet() (not enabled by default).
FossilOrigin-Name: 51f7ee844057086789dcfcdcba7daf45343cae62
2010-01-02 19:02:02 +00:00
dan
3540c1f7b8 Add the experimental FTS3 matchinfo() function. Provides details of the match that may be used for result ranking and other purposes.
FossilOrigin-Name: 37a1de02d1d8a34604f1bee896eaf579d4ba149a
2009-12-22 18:56:19 +00:00
dan
6d67e33776 Mark fts3ReallocOrFree and fts3InitVtab as static. Ticket [ff44d82f3b].
FossilOrigin-Name: a9038306c33c88120d8bef27209d8f0641c85c9b
2009-12-09 05:30:36 +00:00
dan
84db21ec6a Add tests to improve coverage of fts3. Associated bugfixes.
FossilOrigin-Name: f0eac4175aee6c50ee68acc253f76fbe44574250
2009-12-08 19:05:53 +00:00
shaneh
e585b8f05c Updates to FTS3 to correct compiler warnings under MSVC.
FossilOrigin-Name: 37495b55ffbdc2db4482367ac7d8e32d4d71d58e
2009-12-03 06:26:46 +00:00
dan
09977bb9f0 Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.).
FossilOrigin-Name: 30a92f1132801c7582007ee625c577ea2ac31cdf
2009-11-13 10:36:20 +00:00
danielk1977
ee0484c1b5 Add the experimental API sqlite3_strnicmp(). Modify fts3 so that in terms like 'column_name:token' the column_name is interpreted in a case-insenstive fashion. Ticket #3996. (CVS 6950)
FossilOrigin-Name: 4571aa9e9142db465ae8250b0adf27e0a094331a
2009-07-28 16:44:26 +00:00
danielk1977
e932ba260e Fix a double-free that can occur when using the fts3 legacy syntax '-' operator. Add tests for the same operator. Ticket #3960. (CVS 6874)
FossilOrigin-Name: c19d419e8cf94a26d9bb6ad478e84841168a882e
2009-07-10 09:24:43 +00:00
danielk1977
2fe5cb1809 Avoid fts3 crash on (MATCH '""') expressions. Ticket #3717. (CVS 6343)
FossilOrigin-Name: 03679857a320517a7b89e5214e948bce9af896a9
2009-03-12 15:43:47 +00:00
danielk1977
e1d3ac9cd0 Add a comment to fts3_tokenizer.h to make it clear how the xNext() method is supposed to set its output variables. Make sure the output variables of xNext() are only used if SQLITE_OK is returned. Ticket #3604. (CVS 6198)
FossilOrigin-Name: 5b3c075f96be9671d0bcffe928589b211559e835
2009-01-21 17:45:33 +00:00
danielk1977
fc8c9f84ab Fix some problems in the fts3 expression parser with mismatched parenthesis. (CVS 6095)
FossilOrigin-Name: ccfe4580ac7ba9add0e69c786a9a3a43d69b7753
2009-01-01 14:06:13 +00:00
drh
b39187ae89 Additional test cases and cleanup of FTS3 parenthesis processing. (CVS 6094)
FossilOrigin-Name: afac4293000f81410d105a99956605bf7102fa62
2009-01-01 12:34:45 +00:00
danielk1977
5973e6a30b Add pseudo-random tests of the fts3 expression parser. Revise the fix in (6091). (CVS 6092)
FossilOrigin-Name: 11c2d4686197fb3f0d601651d5bbb3492af8f0dd
2009-01-01 07:08:54 +00:00
danielk1977
49b4b4d84a Fix a bug parsing "<expr> AND (abc NEAR def)" in fts3_expr.c. (CVS 6091)
FossilOrigin-Name: d1a6a2edd799d65ff88510df951e909919e35b6b
2009-01-01 04:19:51 +00:00