Commit Graph

369 Commits

Author SHA1 Message Date
drh
707f1c560a Fix a harmless MSVC compiler warning.
FossilOrigin-Name: 6cc023bb29be51847fbbfab95c24fc89993ccdba
2013-11-19 18:17:20 +00:00
drh
2bea7cde6e Fix harmless compiler warnings from clang scan-build.
FossilOrigin-Name: 8d002740bffca2a76d2dfbc1a67293d34f9de9ba
2013-11-18 11:20:50 +00:00
mistachkin
3aa4be39a9 Fix harmless compiler warning.
FossilOrigin-Name: ddacd10105c6df2d3a9d707947e72c62e88212eb
2013-11-12 21:10:02 +00:00
dan
85963f5474 When possible, have FTS use 32-bit operations to read varints from the database.
FossilOrigin-Name: aa7ba302ed13aedde89b5bcbe9119799c0da8a42
2013-11-12 17:46:44 +00:00
dan
d6dd5def4d Fix a bug preventing FTS from correctly processing bracket tokens that are immediately preceded by characters that are neither whitespace or token characters.
FossilOrigin-Name: 49be646cd981f8ff0434cf90d2748afa30260017
2013-11-04 08:56:22 +00:00
dan
a4bbc8b119 Add the "languageid" hidden column to fts4aux.
FossilOrigin-Name: 891df358e51075fb937f34952ce43bf51130b0d0
2013-10-15 19:06:17 +00:00
dan
362d21614e Fix a crash in FTS incremental phrase processing that can occur if the second or subsequent token is much more common in the dataset than the first.
FossilOrigin-Name: 0bf438fc30582a08fddfc3cec49366ee17ae2abe
2013-10-14 20:30:51 +00:00
mistachkin
b084582299 Fix harmless compiler warning.
FossilOrigin-Name: 4b130f88fba216e088f61252bbcdde57ec7ee6a9
2013-10-12 02:33:22 +00:00
drh
4f99189051 Fix various harmless compiler warnings. Change the "warnings.sh" script to
work with STAT4 instead of STAT3.

FossilOrigin-Name: 7df06684ab36bfdad9e9aca6940b7a665c2a0cb5
2013-10-11 15:05:05 +00:00
mistachkin
86f72f0721 Fix compilation issue with MSVC.
FossilOrigin-Name: 36d64dc36f18c166b2c93c43579fa3bbb5cd545f
2013-10-07 21:49:16 +00:00
dan
9211f8a6ce Allow FTS4 multi-token phrases to use a combination of in-memory and incrementally loaded doclists. This allows phrases to (partially) benefit from incremental doclists without disabling the deferred token optimization.
FossilOrigin-Name: f6819c5f3363d358e7ef65fe6978f13991bd44af
2013-10-03 19:27:14 +00:00
dan
eb26672d0e Add a test to check that the new multi-token phrase optimization is actually helping.
FossilOrigin-Name: bc3a2ed5fb2402805928b0002457eebf06f87c47
2013-10-02 08:04:27 +00:00
dan
ff574f4c76 Allow multi-token phrases to load doclists from the database incrementally. This allows queries that feature such phrases to benefit from the "docid<?" optimization.
FossilOrigin-Name: ea543f081d93ed1bf66c21ce2108ec94e349f4c5
2013-10-01 20:02:32 +00:00
dan
73632e37c9 Merge trunk changes with this branch.
FossilOrigin-Name: e294a9c7c525d1da698259c2fef089579bfd1c0b
2013-09-30 18:16:07 +00:00
dan
1f03e609ff Fix a performance problem in the FTS4 auxiliary functions triggered by an OR clause in the full-text query.
FossilOrigin-Name: fa0f2f0e3e79ae653118b901e1cca7725dfaf249
2013-09-30 18:14:45 +00:00
dan
19384380f3 Have fts4 full-text queries consider "docid<?" and similar constraints.
FossilOrigin-Name: 6622424a3a149edd35ba2ba0881aa41b4536417b
2013-09-30 11:42:19 +00:00
dan
2505a5fee2 Fix an FTS4 problem introduced by [361084e1eb].
FossilOrigin-Name: 5f35c8cb8f8b67121c70f98c02c4aa7c25327690
2013-08-21 15:45:27 +00:00
dan
2c18788ffa Initialize a variable in fts3_write.c on the grounds that the argument required to show that it does not require initialization with is complicated. Add an assert() to where.c to silence a clang scan-build warning.
FossilOrigin-Name: d6c4d48a002a6d7057fccc30064ce0b049678f0c
2013-08-19 19:29:50 +00:00
dan
290703091e Add some extra assert() statements to silence a few clang warnings.
FossilOrigin-Name: 1c63e9515b5f5cb9169928b108d002dcd18ad4fd
2013-08-19 18:17:03 +00:00
drh
1f10c28618 Modify several extensions to use the new exported function naming.
Fix some shared library compilation issues.

FossilOrigin-Name: 1e39f85077f1f2b96c3a656c5b6334bafb005908
2013-07-09 12:36:30 +00:00
dan
ed20073b8d Make a couple of implicit casts explicit to fix compiler warnings.
FossilOrigin-Name: 9676280a9387b8bfba82cfc0087318863d05d4f3
2013-07-05 10:46:08 +00:00
mistachkin
049d487e2e Modify several extensions to use the new exported function naming. Fix some shared library compilation issues.
FossilOrigin-Name: f2ab8747825ab5131ffab174aa0ffe5e474f6811
2013-07-04 23:53:56 +00:00
drh
4580cfb93e Cut over the next generation query planner. Increase the version number
to 3.8.0.

FossilOrigin-Name: 0fe31f60cadc5fe5a9d87e110bfaed5fd026cba1
2013-06-26 11:43:18 +00:00
drh
adbae616bd Remove an unused function declaration from the FTS3 source code.
FossilOrigin-Name: 096ae1d8f9a08f92daedece6b0615f4d22b05023
2013-06-21 18:36:44 +00:00
dan
8def92bac3 Add the "notindexed" option to fts4.
FossilOrigin-Name: 8ff2b8f5948ccddce70102e6d68d464c66e4f7ca
2013-06-21 17:30:47 +00:00
drh
37ca04880a Merge all changes from trunk.
FossilOrigin-Name: f2e15b1974e55373b5819e3b2326653f890f409f
2013-06-12 17:17:45 +00:00
dan
5f0b33c2cb Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option.
FossilOrigin-Name: 24fc9d4438a5615dd20af5419456166df83a72ea
2013-06-11 14:22:11 +00:00
dan
f2c9229f73 Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD.
FossilOrigin-Name: 6cfd9af5250029c0d275be027b4208c48954a8a1
2013-06-05 16:17:21 +00:00
drh
73843cac73 Increase the estimated full-scan cost for an FTS3 table.
FossilOrigin-Name: 8d9f29f847f38612447c79a5fb366a9143ef6205
2013-06-04 13:07:44 +00:00
dan
87ddfeb079 When writing to an FTS table, take an exclusive shared-cache lock on the %_segdir table before writing. Otherwise, an xCommit() call may report an SQLITE_LOCKED error.
FossilOrigin-Name: 3cd2da42e9403b1e6243ad53f3f2bbf89c0fb9b0
2013-05-27 10:11:53 +00:00
drh
92692d32ad Fix a couple of harmless compiler warnings in the fts3_tokenize virtual
table.

FossilOrigin-Name: 1fa8c457394c94864f7584e4c893ec09e685fba4
2013-05-08 17:06:28 +00:00
dan
e8dee7d1ed Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this.
FossilOrigin-Name: 0ba67b64de258883e4c43db09e131bb67083855e
2013-05-07 12:16:48 +00:00
drh
acf9c2b555 Add an assert() to fts3_expr.c in order to silence a clang warning.
FossilOrigin-Name: d8dc2c5fb554eb5898ddc8d703c6a360af7fe023
2013-05-06 14:57:48 +00:00
dan
3a01b600d3 Improve the error message issued when an FTS query exceeds the maximum allowable tree depth.
FossilOrigin-Name: f480b1fe6012f36c59cd0525efdc6df74143ccd0
2013-04-29 18:07:37 +00:00
dan
181f4f789d Fix an off-by-one in the code for limiting the depth of FTS expression trees.
FossilOrigin-Name: 72ac73189c3577740a77d2ea2fc7118391c0703f
2013-04-29 17:12:06 +00:00
drh
6f77140f78 Fix harmless compiler warnings in the FTS expression parser.
FossilOrigin-Name: 3c78af8c535e16518f18733325f4cd1df7ec8282
2013-04-26 13:14:19 +00:00
dan
aa29c86e49 Rebalance FTS expressions after parsing to limit recursion during evaluation. Avoid recursion when deleting FTS expression trees. Enforce a limit on the depth of an expression tree.
FossilOrigin-Name: f968d43f80cc2f236e7d09ba1e8278343e2b6976
2013-04-25 20:34:02 +00:00
drh
da8caa0b2d Fix harmless compiler warnings.
FossilOrigin-Name: 1a1cf5aa86734c832d845e07780262a178188d56
2013-04-22 23:38:50 +00:00
drh
4d648c777b Update MSCV makefile for the fts3tokenize virtual table. Clear some
harmless compiler warnings in the fts3tokenize code.

FossilOrigin-Name: 2b3fed09eb89f1e9d6a5f7487ef526b46eb51490
2013-04-22 17:07:56 +00:00
dan
d7a959c694 Add virtual table module 'fts3tokenize' to fts3. fts3tokenize provides SQL access to FTS tokenizer implementations.
FossilOrigin-Name: c5a4b21a15bd8a184380817ff41657acbccb866d
2013-04-22 15:30:37 +00:00
dan
a8f60381a9 Change the fts4aux virtual table module so that fts4aux tables created in the temp database may report on fts3/fts4 tables in any attached database.
FossilOrigin-Name: 546a187f1361dad340ed8f6c28dd89e6c19f9c33
2013-04-12 16:47:27 +00:00
dan
1cb90c668c In fts3, when filtering lists for hits in a specific column, edit the list in place in the same way as it is for NEAR filtering. Fix for [38b1ae018f].
FossilOrigin-Name: f85f9103cffa5c8ba6a63a68beb90817147ba080
2013-03-25 11:38:44 +00:00
mistachkin
48864df97d Many spelling fixes in comments. No changes to code.
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
2013-03-21 21:20:32 +00:00
drh
7b6be6f7bf Fix an uninitialized variable detected by valgrind.
FossilOrigin-Name: ffd1e51490286abfaea68fd4b4b4cb967d87b04b
2012-12-04 00:53:08 +00:00
dan
bba44d0d1b Fix a problem causing the "number-of-documents" field maintained by FTS4 to be set incorrectly.
FossilOrigin-Name: e38fb02d5ea5daa6992df4dfbbeec92bf7b525f6
2012-11-27 15:56:38 +00:00
dan
e62c0694de Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition.
FossilOrigin-Name: 33f4f9817e8b3cb97ce02a4c49bd586332da37b2
2012-10-25 19:25:39 +00:00
drh
8749c1831a Make sure substructure elements have proper alignment in the ICU tokenizers
of FTS2 and FTS3.

FossilOrigin-Name: aaa2d9b0db74d8452d9294de17cff786ab4ec7c8
2012-10-19 02:10:53 +00:00
dan
1276cd0ba8 Add a comment to fts3_snippet.c.
FossilOrigin-Name: b5f5ba3103e6fcf117fb69d5379fa70f691a652d
2012-10-17 20:28:52 +00:00
dan
6c42aa4a0e Initialize variables used as outputs of custom tokenizer method calls in fts4.
FossilOrigin-Name: 514505a84bcc677518d60a42896598f85ed78265
2012-10-17 20:15:10 +00:00
mistachkin
d5578433ff Fix all known instances of 'repeated the' style typos in comments. No changes to code.
FossilOrigin-Name: 9b19b847533f944f289d93dcdba29c0d67bf251c
2012-08-25 10:01:29 +00:00