Commit Graph

208 Commits

Author SHA1 Message Date
dan
09643ab745 If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms.
FossilOrigin-Name: b5bdc639898ee22eebedeb560810e94e74de8aa4
2011-05-24 18:49:45 +00:00
dan
133d7dab17 Add extended return code SQLITE_CORRUPT_VTAB. Returned when the tcontents of the sqlite tables used internally by a virtual table module are invalid or inconsistent.
FossilOrigin-Name: 8844e8bfb87314fb40ecb92705e8fff88f72bb38
2011-05-17 15:56:16 +00:00
dan
62d6c7e32b Fix a couple of compiler warnings in the FTS code.
FossilOrigin-Name: 1a113359705d307efa1833b1dfa5542b045dae43
2011-05-04 16:30:05 +00:00
dan
3480a01dad Add documentation for the newly introduced sqlite3_vtab_config() and on_conflict() API functions. Test that encountering an SQLITE_MISMATCH in fts3 does not corrupt the full text index.
FossilOrigin-Name: abdd70ae0424ccadb7edaf16e970c78b5257d23c
2011-04-27 16:02:46 +00:00
dan
a311b80392 Extra tests for fts3. And fixes for conflict-handling related problems in fts3.
FossilOrigin-Name: fb4a355871d9482ccb28b6ba03b842b3cc87b696
2011-04-26 19:21:34 +00:00
dan
b061d058cb Add support for on conflict clauses to fts3/fts4.
FossilOrigin-Name: 6d2633a6d0a9bb88fb1a6adac0827dc51df2d4d2
2011-04-25 18:49:57 +00:00
dan
355c53b6ef Avoid passing NULL to the xOpen method of an FTS3/4 tokenizer.
FossilOrigin-Name: 0dd09fc034c127718366d3a3183e367d2f9fd82d
2011-04-19 06:43:22 +00:00
dan
4c600ac389 Change fts4aux to (additionally) report on term frequency in individual columns of an fts table.
FossilOrigin-Name: 3996f92a9aca9ac2628be003eca83c8f954c71de
2011-02-04 18:56:25 +00:00
dan
20d5f354e9 Optimize handling of equality and range constraints on the "term" column of an fts4aux table.
FossilOrigin-Name: 386701ded2bcee5309ec9f285d94e6eb1eade193
2011-02-02 17:30:43 +00:00
dan
0bfda98155 Fix a case in fts4 where a corrupt %_stat table could lead to a crash.
FossilOrigin-Name: 4ade96ce974244fc34bb97713d3cba10e3d33056
2011-02-01 17:55:48 +00:00
dan
a240fd01b6 Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options.
FossilOrigin-Name: b010ddcc52889160af2183a33c5f483bb0ae91b9
2011-02-01 16:34:32 +00:00
dan
a8ab60c648 Fix a couple of crashes in fts3 that can occur if the database contents are inconsistent.
FossilOrigin-Name: 811e12cddfb3246c6cf3d5085bd9b72b12e05550
2011-01-13 11:20:04 +00:00
dan
86596e552e Fix a segfault that can occur in matchinfo if an fts4 table contains mostly zero-length documents. Specifically, if the table contains more rows than it does bytes of text.
FossilOrigin-Name: fe9047668eaaf76e7aa1ef1f32dec7c7c4226e45
2011-01-13 10:58:26 +00:00
dan
b9a93d70b8 Reduce the number of calls to sqlite3_realloc() made by fts3 when querying for position information of a term prefix.
FossilOrigin-Name: 7088d9450f403f12f67eed558e368573101245d6
2010-12-24 15:49:27 +00:00
shaneh
556f6bbc92 Fix some warnings under MSVC in fts3 module.
FossilOrigin-Name: c7771c0b22f2b45a47070cf84b9ecf1011e40404
2010-12-01 15:36:00 +00:00
drh
6ea28d6d84 Fix various compiler warnings.
FossilOrigin-Name: c412f61229b6ab1ac90b932afd56f7c5e3ba1cfe
2010-11-26 16:49:59 +00:00
dan
f9376540fa Add code for the matchinfo 'longest common substring' feature.
FossilOrigin-Name: 71011a4f9baf09ec6935ad591145252bf3c286ed
2010-11-24 19:26:18 +00:00
dan
ad3acbbfb3 Remove some unused code from fts3. Add tests to fts3matchinfo.test.
FossilOrigin-Name: ae40b34cf7c24c9601bdfb5cbe5b20f05a376ea8
2010-11-24 11:51:56 +00:00
dan
1e66e40eb9 Experimental changes to fts3 function matchinfo().
FossilOrigin-Name: 9cf0f2b76bc68c168e3fa861b7235f384db21d38
2010-11-23 19:16:47 +00:00
dan
af4c214e1f Changes to allow FTS4 tables to be created without the underlying %_docsize table (in order to save space).
FossilOrigin-Name: 31989b18f53d97eddfb39660ef04fbf9463583e0
2010-11-02 17:41:52 +00:00
dan
cfddb09e3b Test cases and minor changes to make fts3 more robust in the face of a corrupt database.
FossilOrigin-Name: b770290561f5450e4d985ca0050ef5eb01657c80
2010-10-30 15:21:13 +00:00
dan
797f3ee87c Add extra test cases and changes to fts3 to avoid crashing on a corrupt database.
FossilOrigin-Name: 252f0e457d3e33404df87d3e6c44ede61b78319c
2010-10-29 18:45:10 +00:00
dan
8aaa252d8c Fix a buffer overread in fts3 that can occur if the database is corrupt.
FossilOrigin-Name: 84194c4195d7144ff7f9cedcdc74fdd908f3bfcd
2010-10-27 16:52:27 +00:00
dan
8c30711208 In fts4, store the total number of bytes of for all records in the table in the %_stat table.
FossilOrigin-Name: 941647d121ac60e2eabc998cfe79b157fb918d7e
2010-10-27 10:55:54 +00:00
dan
378d0ab97b Fixes for the matchinfo() function related to FTS4 common token handling.
FossilOrigin-Name: deb80eac9112d21835dfd3cee08ed8f09d975bf7
2010-10-23 19:07:30 +00:00
dan
789cb8f5cf Add missing comments and fix compiler warnings in new FTS3/4 code. Other minor fixes too.
FossilOrigin-Name: 1c9c70fec3c88319f7b2efe5316694a6ce0ab1a5
2010-10-22 16:44:39 +00:00
dan
4e76cc3650 Updates to FTS4 to improve performance and make more accurate cost estimates for prefix terms.
FossilOrigin-Name: d0a450ce78e99f55c862f26f9332786660007a0a
2010-10-20 18:56:04 +00:00
dan
4f7c5e684a Experimental changes to fts4 to try to selectively avoid loading very large doclists.
FossilOrigin-Name: 5ae0ba447a561e3b6637b52f9b83a9fc683d2572
2010-10-19 14:07:59 +00:00
dan
8786f7f589 Have all FTS3 queries obtain a read or write table-lock at the shared-cache level before doing anything else.
FossilOrigin-Name: 018e82c775d0fb8c0d90cddf7a87c20c8c8172a9
2010-09-17 15:28:41 +00:00
dan
3b6f4c06ac Rewrite a couple of queries used internally by FTS3 to take advantage of the "SELECT max(x) FROM ..." optimization.
FossilOrigin-Name: b7e42ae77443b33e0ab83265064830594094dc7c
2010-03-19 06:59:19 +00:00
shaneh
84f4b2f237 Minor changes to silence some compiler warnings with MSVC.
FossilOrigin-Name: 4927c62d6214d93e582bf1266215ed3519630c15
2010-02-26 01:46:54 +00:00
drh
b4a1fed2ea Add a new full-text search variant that tracks the total number of documents
and document sizes, to make ranking search results easier.  Currently
called FTS4.

FossilOrigin-Name: 1b6e6094c88214e02c9e3638932997ac20bfe413
2010-02-03 19:55:13 +00:00
dan
8718183ea5 Fix a bug triggered by optimizing an FTS3 table when there are no segments on disk but pending terms in the hash table.
FossilOrigin-Name: a3b50e4f80ca2dacc1f72435b8c72d55ffdb64e7
2009-12-29 10:32:36 +00:00
dan
971742058f Fix a broken assert() in fts3_write.c. Also fix a couple of compiler warnings in fts3 code.
FossilOrigin-Name: a730a05b39c45138765df98d34b9db1ab4e72152
2009-12-14 14:49:57 +00:00
dan
55be7449f9 Further fts3 coverage tests.
FossilOrigin-Name: d2a8c0f683271f5fb8c9badfb13e4e46fd78db71
2009-12-12 16:04:32 +00:00
dan
45bcd6c274 Extra tests for coverage of fts3 code.
FossilOrigin-Name: eee921a99e69a9cd868a89de620bf47c4e26e4b5
2009-12-12 13:16:09 +00:00
dan
f5fff2a7b2 Add coverage test cases for fts3.
FossilOrigin-Name: 8fcb0478c82507403165719724b62a308cb83b57
2009-12-12 09:51:25 +00:00
dan
601cd9aa24 Change the fts3 test interface used to configure the advisory node size paraemeter.
FossilOrigin-Name: 87fc0ce151c89beb5a43b65c0b1706f340c0c173
2009-12-11 16:03:45 +00:00
dan
4f887abc4c Rationalize some code in fts3 used by optimize operations, queries of the pending-terms hash table and segment merges. Add the "INSERT INTO tbl(tbl) VALUES('optimize')" syntax.
FossilOrigin-Name: 29476da353df4c67fe744c1c5f466ba5b9c1a54b
2009-12-11 12:29:04 +00:00
dan
b893721a45 Changes to fts3 to avoid flushing data to disk within a SELECT statement.
FossilOrigin-Name: 48c0db0eb2d134bb302bb5eca6beb0ec46736257
2009-12-10 16:04:25 +00:00
dan
18ff7fa6b4 Improve test coverage of fts3.c.
FossilOrigin-Name: 56b6432f8622d53ffd3a4d9a2244114f8531ed71
2009-12-09 14:39:41 +00:00
dan
84db21ec6a Add tests to improve coverage of fts3. Associated bugfixes.
FossilOrigin-Name: f0eac4175aee6c50ee68acc253f76fbe44574250
2009-12-08 19:05:53 +00:00
dan
e2e5145441 Fix an incorrect assert() in fts3.c. Add further fts3 tests.
FossilOrigin-Name: 75863c2d55e0801add5b8dcf88d575c5c870af04
2009-12-03 17:36:22 +00:00
shaneh
e585b8f05c Updates to FTS3 to correct compiler warnings under MSVC.
FossilOrigin-Name: 37495b55ffbdc2db4482367ac7d8e32d4d71d58e
2009-12-03 06:26:46 +00:00
dan
bc7c039ce2 Clear the Pager.dbModified flag when unlocking the database. Assert that it is clear when locking the database.
FossilOrigin-Name: d17ec16b7c5051c904c09580a856593b2fb85edc
2009-12-02 14:44:32 +00:00
dan
3acb07d6c3 Open a savepoint within the FTS3 optimize() function.
FossilOrigin-Name: 4924fbb244bd1b7103e29e045812cb1c4d2d81c8
2009-12-01 13:48:13 +00:00
dan
612b1d5cd0 Fix a segfault that can occur when querying an empty FTS3 table. Also restore the rowid/docid conflict handling to work as it did in version 3.6.20.
FossilOrigin-Name: c022f66b5a65aa54d5ebd55cfe941118a2042280
2009-12-01 12:00:22 +00:00
drh
0a62730d3f Updates to snippet() and offsets() functions of FTS3 so that they work
sanely following an OOM fault.

FossilOrigin-Name: b939a37a8ce296785a300e79ab9d3d87ad91343f
2009-11-28 21:33:21 +00:00
dan
81fa6dc319 Fix a bug introduced by the fts3 refactoring (segfault when creating a table with zero module args). Also a fix to handle an OOM error.
FossilOrigin-Name: eada284bc10cafcab9beb3473bb0c70b3b4de2f9
2009-11-28 12:40:32 +00:00
dan
7eee299243 Add some missing comments to fts3 files. No source code changes.
FossilOrigin-Name: b6402b2065b844acb3f1bb94ad964568706bcb86
2009-11-21 03:03:21 +00:00
dan
e6828f5503 Merge leaf accidentally created by [1c4984c62f].
FossilOrigin-Name: cae949ce971ca216e0f8880b2f93866619fa05be
2009-11-20 10:23:12 +00:00
dan
d1414c58e5 Improve comments and other things in fts3_write.c.
FossilOrigin-Name: 1cf0e3cc14bad22867e740736c2886dc1c4a48dc
2009-11-20 05:05:19 +00:00
dan
d313865550 Minor optimizations to fts3 code.
FossilOrigin-Name: b456eacbbb16513d1b27e90015ea58a6dc92cc3b
2009-11-20 02:24:15 +00:00
dan
16708c4a0d Fix some fts3 related issues with the autoconf and amalgamation build systems.
FossilOrigin-Name: 3b17924754343c0163464dabf01a9c46ffccef28
2009-11-19 15:25:25 +00:00
dan
5dc842ddf7 Fix problems introduced into fts3 as part of the refactoring.
FossilOrigin-Name: fa0998e19d984ee57f4f506c34eb858026cc49c3
2009-11-19 00:15:27 +00:00
dan
bd61689382 Add some missing comments and fix some other issues in fts3 code.
FossilOrigin-Name: 2fe579e778b75fbf503c02e01e5424c1926f2b49
2009-11-18 15:35:58 +00:00
dan
f13b704ee6 Improvements to the way fts3 reads the full-text index.
FossilOrigin-Name: 45c051e78651d8204c17cecdda2bde705698881f
2009-11-17 12:52:10 +00:00
dan
09977bb9f0 Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.).
FossilOrigin-Name: 30a92f1132801c7582007ee625c577ea2ac31cdf
2009-11-13 10:36:20 +00:00