Commit Graph

661 Commits

Author SHA1 Message Date
dan
495ce22d49 Fix a "shift exponent is too large" usan error that could occur in the fts3 snippet() function when a query phrase contains more than 64 tokens.
FossilOrigin-Name: 6d2c7ac4ecfc1ec408cb1820985a70650d7cc5613b4b8fd1829f68ea48fce6a3
2024-04-09 19:23:18 +00:00
dan
adea051707 Ensure the fts3 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
FossilOrigin-Name: 1bdb8cbaf7e733e0731de2ecd7a05ddf44db5d8595dcc7aaf0927ed0aa5b33d3
2024-02-01 15:21:55 +00:00
drh
31ebe65b5e Fix harmless "unused parameter" compiler warning in the new fts3IntegrityMethod
implementation.

FossilOrigin-Name: bb1fe53a97672fc868e3f3897162ea0d714dc1da95187f9c7e35255049f2b0e7
2024-01-16 16:14:52 +00:00
dan
75f9ba32bd Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases.
FossilOrigin-Name: b855886c4ccce0745af6957943e77be18949722f09821688725d546d3d79b4fb
2024-01-16 16:05:21 +00:00
drh
9f20bde65a Enhance the new xIntegrity method of the sqlite3_module object with new
parameters that provide the name of the table being checked and
a flag to indicate a "quick_check".  Based on feedback in
[forum:/forumpost/965c0d02ea|forum post 965c0d02ea].

FossilOrigin-Name: bc8afa3f15954bb35f65dbf940bf069de5e14d333036676c24430cf17b658d05
2023-10-25 10:37:11 +00:00
drh
c0ba6a97ec Fix various harmless scan-build warnings.
FossilOrigin-Name: 54be9af4469d7e31ee852f67e5aa32996557c10de654a60103fd165d2fedf311
2023-10-24 11:06:44 +00:00
drh
45b2e946cc Improved error messages for the xIntegrity method in FTS3/4 and FTS5.
FossilOrigin-Name: df69abc0823f4dd9c051f8d5993d2f447ee0dadab5cef73d670107b1071db255
2023-10-23 13:46:22 +00:00
dan
ef6bf1bbe5 Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption.
FossilOrigin-Name: 7f41d7006db4225cf9b3d197d3a76842778669ac079e76361214a8023c9976e6
2023-10-21 18:12:07 +00:00
dan
ee3c55471c Fix an fts3 problem caused by reducing the page size using the undocumented "nodesize" option, then running an incremental-merge.
FossilOrigin-Name: 2875dcb1ac86db2704ad377d719f308c8837e196b49c98b20ceda1324f2d27ea
2023-10-19 18:23:17 +00:00
drh
4b136ed70a Enable SQLITE_VTAB_INNOCUOUS for FTS3, FTS5, and RTREE.
FossilOrigin-Name: f34c533b6c1f8ef3f69c75c1039406c12751cdde1fbdcb4d9776b24455facf8a
2023-10-12 18:46:27 +00:00
drh
1935887a68 Ensure that all fields of static sqlite3_module objects are explicitly
initialized, in order to hush-up nuisance compiler warnings.

FossilOrigin-Name: f3b3d712d6e58b1cb8fdebd2b6b3125080b6b3ac8c7c849a8cc1e5e778d62fe7
2023-10-06 12:51:05 +00:00
drh
dd69129f2a Fix a problem with the new xIntegrity method for virtual tables, and also fix
a bad assert() in FTS3 that was found by the new xIntegrity method.

FossilOrigin-Name: 52bbf44f2d9addc2b5f68b0fe33542470852310ce3a283e2c7ff4c52831d0ed1
2023-09-07 02:13:01 +00:00
drh
d5ab4dd9e4 Change the xIntegrity virtual table method signature so that it returns
an integer error code and writes the error message into a parameter.

FossilOrigin-Name: f1d4024a8ca06cf954aaf1f612684d1a5d28492bde757695db3f22c50c649709
2023-09-06 14:00:01 +00:00
drh
961c2a9f36 Add the xIntegrity method to the sqlite3_module object. Implement this
method in RTREE, FTS3/4, and FTS5 so that "PRAGMA integrity_check" also
verifies the correctness of shadow tables associated with those virtual
tables.

FossilOrigin-Name: 17bede8cdefd968210dd8a5a2617acbe12ba2c99fdd5e88c5def8665e7bec2d7
2023-09-06 12:52:00 +00:00
drh
d97e0d0e38 Initialize a local variable in FTS3 to avoid a GCC compiler warning.
FossilOrigin-Name: 7ec4ab327decd6a5ee5e6a53f1489e17e0cdbb297945f9acc532b47d052eb7a9
2023-06-12 18:22:34 +00:00
dan
9b532d1e1b Avoid a buffer overread in fts3 that could occur when processing a corrupt record.
FossilOrigin-Name: 02ac2297abee6af64c8df230b42b07f21cff4565d7e315860b2396a7c0c556ca
2023-03-18 16:12:27 +00:00
drh
e3d71c078b Fix a harmless compiler warning in FTS3.
FossilOrigin-Name: 68d02d3c2ab1b4afff2d3a71516ceee404fad0d2b717369ad732832c4f9b19a8
2023-01-25 21:28:30 +00:00
dan
a0fe1e7d13 Fix a problem with fts3 auxiliary functions and one or more NEAR expressions ORed together.
FossilOrigin-Name: de4690a10ad4631e7452ccbb05b177a821d9dda387a854d216a6c54c7a189ead
2023-01-25 15:45:45 +00:00
dan
0fa2bbf9e1 Fix another problem with fts3/4 auxiliary functions and NEAR expressions that consist entirely of deferred tokens.
FossilOrigin-Name: a8c91c132f6157b7e3649f57a799984b1d7f8a18fd434515c875617d4195db29
2023-01-25 13:42:55 +00:00
dan
e35d1e1e6a Fix a problem with using fts3 auxiliary functions with expressions like "E AND ...", where E is a NEAR expression that consists entirely of deferred tokens.
FossilOrigin-Name: 39bfae4c4698a13e07c4a0725f2790955e03b601fe64e17a000c691def1bdcb8
2023-01-24 11:24:28 +00:00
drh
6319a8003c Make the legacy FTS3 code more robust against integer overflow during
memory allocation.

FossilOrigin-Name: 5517bc50988b5339c2fd071b29de1b5ca03037b0b635c3b112cf7108fab54d5f
2022-09-27 01:53:05 +00:00
dan
8e658f0a13 Avoid having fts3 read uninitialized values when processing deferred tokens.
FossilOrigin-Name: 98729c5fa6c4c418ca9779c35e6874c930e5cee5a90b5eb32b284a0f3762269e
2022-08-05 17:16:00 +00:00
dan
40e3fa3035 Fix a memory leak in fts3 that could occur when processing a corrupt database.
FossilOrigin-Name: d74f6f6d5136995b8bf900eb671e4b15ca81e03cc1ab5b7a1aa43dc4f3617760
2022-07-15 15:08:48 +00:00
dan
b1c034b2f1 Update some faulty assert() statements in fts3.
FossilOrigin-Name: b072851be1a5e49441469dbfa4580e132ebb1d8bc2de98fe1aa67bb9c7cf13a3
2022-07-15 11:34:23 +00:00
dan
3dfab26028 Fix a problem in fts3 to do with deferred tokens and OR expressions.
FossilOrigin-Name: d0bfe5c574dbc467ba97d96306d78a62a55b78dbbbe4d25aa962db6aa40406f6
2022-07-11 14:26:38 +00:00
dan
181d75ef9e Fix a 1-byte overread in fts3 that could occur when processing corrupt records.
FossilOrigin-Name: f1dfbc4f7452154ca5190ac22e17fdabebb92833967b11f7dae21b8d3a5857bd
2022-06-03 13:52:53 +00:00
drh
412a59f56e Change an assert() in FTS3 into assert_fts3_nc(), since it is a harmless
condition that can occur when processing a corrupt database file.

FossilOrigin-Name: e773d6219bded9a5dfa9930882682dadaa7f1c6be344dbde904c9477a89f2d2c
2021-12-29 04:10:03 +00:00
dan
62e9c352eb Remove old script ext/fts3/mkfts3amal.tcl, as it doesn't work.
FossilOrigin-Name: 9e33ee24b3793b3d7faf7421279d16217f8b09b1e92bfdc771f902f2f1e1b7d6
2021-12-20 17:00:42 +00:00
dan
2999068d2c Fix fts3 and fts4 so that they can still be used even if the other is dropped by a call to sqlite3_drop_modules().
FossilOrigin-Name: a0a8d6c9f0e91996003626e4f80dfac278e3c5bddd9ace5d442061e6c95e49dc
2021-12-16 15:29:49 +00:00
drh
156d44104a In the previous check-in, do not overwrite the SQLITE_CORRUPT signal with
an SQLITE_OK before exiting fts3auxNextMethod().

FossilOrigin-Name: 396688ae5c10b9e7df1c76a3c6f548abf7a7ddad50c25fb23e194201dbb2e857
2021-11-22 12:59:44 +00:00
dan
bb0c54283d Fix a crash in the fts3aux code that could occur handling corrupt records.
FossilOrigin-Name: cdcd65b89ecffd380483404f5b1379beffd7ff107acaa5bc1185651cfacaa220
2021-11-22 08:12:14 +00:00
drh
608af68ad3 Use the FTS_CORRUPT_VTAB macro rather than a direct call to
sqlite3Fts3Corrupt() in [2fad2a89527757b3] so that the build works
in all cases.

FossilOrigin-Name: 029c59cdf9e7dbb431f5d110bc69c3597458edc9b6b009b2e91422de705a19fa
2021-11-05 12:34:06 +00:00
dan
3555274419 Fix a crash that could occur in fts3 when processing a corrupt database.
FossilOrigin-Name: 2fad2a89527757b3956538a6bf91ebebe9c483f22f94e889e8b50c58f5100c5c
2021-11-04 18:04:55 +00:00
dan
fb8e71c584 Fix a case in the fts3 offsets() function where a corrupt database record could lead to dereferencing an uninitialized pointer.
FossilOrigin-Name: 7b7d31a6153b1505288eb3e849d0d9ef9e88e961c7b2f918ef5582fd77990f6d
2021-10-20 11:40:34 +00:00
dan
dee0359ddb Fix a use-after-free in fts3 that could follow an optimize() in the middle of a SELECT statement when the hash table is non-empty.
FossilOrigin-Name: 8124b2ceb71447157283b16f9a7fa6e250003b9ddf1c1b45f91db64aed79b5da
2021-10-12 11:30:07 +00:00
drh
7d4c94bc0f Fix harmless static-analyzer warnings.
FossilOrigin-Name: 32f33f356931242b59c01b5df7e180941231e3d9c91577ad33aa4a01e9d59092
2021-10-04 22:34:38 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5.
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
2021-10-04 18:21:14 +00:00
drh
c59ffa8c1e Fix harmless static analyzer warnings.
FossilOrigin-Name: 1ebcde72e2046dffaa408d2a5a4a5bbf35fdf4b096e0e088dcffd9360effdaa7
2021-10-04 15:08:49 +00:00
dan
822810b95a Avoid a buffer overread in fts3 that could occur when handling corrupt data structures.
FossilOrigin-Name: 45f459d2fa4be97d9bbb970efbc0b5d40efaf93f52ed111fd0fcdc572c24327b
2021-06-08 12:15:56 +00:00
dan
ec73546103 Fix a buffer overrun in fts3 that could occur when handling corrupt records and '^' queries.
FossilOrigin-Name: 9d0b6b0f42a47a3892ebc765250756fb8b844e8399d992a8b65f55af3800ea06
2021-06-07 15:28:33 +00:00
dan
4fd82a2a78 Fix an undefined-integer-overflow problem in fts3.c.
FossilOrigin-Name: a0bf931bd712037e44e8d7cac3c00a6715c1b451f222bad3184d3ecab1a4c7f4
2021-05-05 11:47:34 +00:00
dan
6c68e55689 Fix another undefined behaviour in fts3 that could follow an OOM error.
FossilOrigin-Name: fac3ff857c4abdb44f59849af80d93a5bd4b64f088e5164fb5ad0be8d14748b7
2021-04-16 17:09:58 +00:00
dan
7f2eb8fff2 Avoid signed integer overflow when finding snippets in fts3 by using 64-bit integer offsets.
FossilOrigin-Name: 4cc09a872f627f4a2b94345bef07cd49c3ec3627f8d78c1eb091741cdb4ec0b3
2021-04-16 16:55:28 +00:00
dan
11a6e9d9f0 Fix an assert() in fts3 that might fail when handing corrupt records.
FossilOrigin-Name: d0e2d6742f314f71562e404980aad20de6b17e0b04c08582c18bbc4fd0360751
2021-04-16 11:25:37 +00:00
drh
6af2a728e4 Fix an obscure memory leak in FTS3. dbsqlfuzz 26a51001beeff35649d2e328b05ec06f93c577a5
FossilOrigin-Name: be7cd9ebb1ba360c60469073b2e526cb1cceea702e43fcc07f47b47986908c8f
2021-04-08 13:41:20 +00:00
drh
e7e7481a29 In FTS3, if the xBegin method fails to start a transaction due to an OOM
error, then make sure that the virtual table is left in a consistent state.
dbsqlfuzz 85ef48423fda4eef79a551654445d8ef3537a590.

FossilOrigin-Name: dd553578333f49950a197fe894d02e08cce011c9ae1a7270604186af204dcf6d
2021-04-07 12:59:09 +00:00
dan
acbae3ba10 Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt.
FossilOrigin-Name: 004559544e661f9afa60306c4ac6842c29a8630b3663d62aa0cdc637f176932e
2021-03-29 20:04:31 +00:00
dan
3fb7bd5ee0 Fix sanitizer complaint in fts3 code.
FossilOrigin-Name: 424b39eaeb578f85e58214af4b467e6e363431308ab7a327774e62fa70d3c2db
2021-03-10 20:04:21 +00:00
drh
4bb4293b2a Limit parenthesis nesting depth in FTS3 expressions to
SQLITE_MAX_EXPR_DEPTH.

FossilOrigin-Name: 5ddd8032ef9ecd5b53909d304c8f1375a72f270fd7810964cb4e385e25bedd4f
2021-03-06 15:13:26 +00:00
dan
fa9011ec70 Fix a crash in fts3 that could occur when running a NEAR query on a corrupt database.
FossilOrigin-Name: 609d94996324f3d3c12bb8cc04a2527d1d86c35cdb2267b5b34053c961158986
2021-03-06 13:28:07 +00:00