Commit Graph

208 Commits

Author SHA1 Message Date
drh
6319a8003c Make the legacy FTS3 code more robust against integer overflow during
memory allocation.

FossilOrigin-Name: 5517bc50988b5339c2fd071b29de1b5ca03037b0b635c3b112cf7108fab54d5f
2022-09-27 01:53:05 +00:00
dan
181d75ef9e Fix a 1-byte overread in fts3 that could occur when processing corrupt records.
FossilOrigin-Name: f1dfbc4f7452154ca5190ac22e17fdabebb92833967b11f7dae21b8d3a5857bd
2022-06-03 13:52:53 +00:00
dan
dee0359ddb Fix a use-after-free in fts3 that could follow an optimize() in the middle of a SELECT statement when the hash table is non-empty.
FossilOrigin-Name: 8124b2ceb71447157283b16f9a7fa6e250003b9ddf1c1b45f91db64aed79b5da
2021-10-12 11:30:07 +00:00
drh
7d4c94bc0f Fix harmless static-analyzer warnings.
FossilOrigin-Name: 32f33f356931242b59c01b5df7e180941231e3d9c91577ad33aa4a01e9d59092
2021-10-04 22:34:38 +00:00
dan
822810b95a Avoid a buffer overread in fts3 that could occur when handling corrupt data structures.
FossilOrigin-Name: 45f459d2fa4be97d9bbb970efbc0b5d40efaf93f52ed111fd0fcdc572c24327b
2021-06-08 12:15:56 +00:00
dan
ec73546103 Fix a buffer overrun in fts3 that could occur when handling corrupt records and '^' queries.
FossilOrigin-Name: 9d0b6b0f42a47a3892ebc765250756fb8b844e8399d992a8b65f55af3800ea06
2021-06-07 15:28:33 +00:00
dan
11a6e9d9f0 Fix an assert() in fts3 that might fail when handing corrupt records.
FossilOrigin-Name: d0e2d6742f314f71562e404980aad20de6b17e0b04c08582c18bbc4fd0360751
2021-04-16 11:25:37 +00:00
dan
c6920cdb0c Fix a case where fts3 was erroneously reporting corruption.
FossilOrigin-Name: e6c7683c7503ac743d1d476c60c31f887b7ad829e26e812c25acdd4366044db8
2021-03-03 16:00:26 +00:00
dan
1e0cfd7184 Fix a case where FTS3/4 could pass a NULL pointer to memcpy() when handling a corrupt db.
FossilOrigin-Name: 68bb541a39833d7d4bf41aa91cb6cd9c98757d1fc8236299d09db7e9b14d8ec9
2021-02-20 19:22:32 +00:00
dan
7db1ed50b8 Fix handling of another corrupt database case in fts3.
FossilOrigin-Name: ccff8cb8267d4c5605484f7a35c1836937f20b3d6879fe84cd84dc24bbbffc77
2020-07-09 21:29:34 +00:00
dan
f488bc1147 Avoid a potential buffer overread in fts3 when processing corrupt records.
FossilOrigin-Name: 4d0cfb1236884349168f8e2ec5e18c0232965148af78615e0d5c9b0e13a35422
2020-06-30 15:32:12 +00:00
dan
30b5db1261 Fix a problem that could cause an infinite loop in the fts3 'merge' command.
FossilOrigin-Name: be545f85a6ef09cc6c762f7d2ab7a0b3adf5590c3fbdc9903e6b5b5cec6e823f
2020-06-26 20:41:18 +00:00
dan
fc7f31742d Fix an integer overflow in fts3 causing a usan error.
FossilOrigin-Name: e256f85289a78e629acdf83e5bf1f8df2a0ffb3d559738eb9e49db6c228dc8c0
2020-04-22 11:11:17 +00:00
dan
7576a68c8c Improve corruption detection in fts3 shadow tables earlier in order to prevent an assert() from failing.
FossilOrigin-Name: a9ec8c8f80a59badabb0afdb4189f0fd2934f936530d4151de395b3a7e7c1f1f
2020-04-16 11:35:27 +00:00
dan
ec8e689a20 Avoid an undefined integer overflow in fts3 by detecting data structure corruption earlier.
FossilOrigin-Name: 86e98ddc19470410ccc6d2cf4ad56ef0bc5a23b7fbe6331b8cae374689f54529
2020-03-20 20:18:49 +00:00
dan
fcbc673752 Fix some fts3 problems found by usan.
FossilOrigin-Name: fb7ccf61bed8d862986eda7096bec3df5947f1d1c88f27e7d9c8acf420f40c50
2020-02-06 10:55:10 +00:00
dan
e455e3acf0 In fts3, avoid making a very large memory allocation if the merge-hint record is corrupt.
FossilOrigin-Name: 9add58fe9688d5c1f675736e7f1c6ff7831d2ff6fdeb884a2e62c1bf9536ee50
2020-01-03 20:16:32 +00:00
dan
5be1b20aec Fix problems in fts3 with processing corrupt records and undefined integer overflows.
FossilOrigin-Name: 3b873029ef1903f76442445343af3786cd59ee66b0e5e0e6b76a0b6b1b5eb88a
2019-12-14 14:15:21 +00:00
dan
8aa706e748 Fix a buffer overread that could occur in fts3 with corrupt %_stat records.
FossilOrigin-Name: e01fdbf9f700e1bd9dd5283c65547d10d26ce4f4506d3cfef9e1087aecdc2305
2019-12-04 03:46:50 +00:00
drh
51690f26bc Fix to check-in [5d9a369301a65f32] so that it compiles without SQLITE_DEBUG.
FossilOrigin-Name: fefe2d046725254367f9f82a8c8cb137e8b75dbbf4bc48debf0649479ff2f0c2
2019-11-26 11:10:59 +00:00
dan
597f1eb195 Fix an assert() in fts3 that could fail due to corrupt database records.
FossilOrigin-Name: 5d9a369301a65f320a0696fcf1f062ca5976ef34350590c07aecf5335c66d872
2019-11-26 02:03:16 +00:00
dan
70d1a1a3ed Further improve detection of corrupt records in fts3.
FossilOrigin-Name: a0f6d526baecd061a5e2bec5eb698fb5dfb10122ac79c853d7b3f4a48bc9f49b
2019-11-20 13:31:52 +00:00
drh
13a8f203d9 Further improvements to shadow table corruption detection in FTS3.
FossilOrigin-Name: e35d8c76aae59f57cc5193f79b21b4298029bea78e1aab7af67432162ce63e00
2019-11-18 12:04:17 +00:00
drh
feb5dcf1a6 Better detection of corruption in the %_stat and %_docsize shadow tables
of FTS3.

FossilOrigin-Name: 1e449687881f4d388e54a0e51bcabba41ab10cf7e596ff65e31e88a23c70d497
2019-11-17 02:41:06 +00:00
drh
fd6bf04bba Remove a reachable NEVER() in FTS3.
FossilOrigin-Name: 8bd75bf636f72f32d66c6c38e1918f27daf2f13290f00a001f41d50838bbda47
2019-11-17 00:08:24 +00:00
drh
c72f2fb7fe More improvements to shadow table corruption detection in FTS3.
FossilOrigin-Name: 51525f9c3235967bc00a090e84c70a6400698c897aa4742e817121c725b8c99d
2019-11-16 23:47:40 +00:00
drh
94febb3a77 Improved detection of corrupt shadow tables in FTS3. Enable the debugging
special-inserts for FTS3 for both SQLITE_DEBUG and SQLITE_TEST.

FossilOrigin-Name: 04b2873be5aedeb1c4325cf36c4b5d180f929a641caf1e3829c03778adb29c8e
2019-11-16 21:40:49 +00:00
dan
9930cfe878 Fix a problem in fts3 causing it to report corruption if a doclist contains consectutive rowid entries that differ by more than 2^63.
FossilOrigin-Name: 0b0a3048f9df88bbc8979f2ac5247bee7d7bc1e585faca252610d2fcbe36d51d
2019-10-28 13:54:59 +00:00
dan
b2075011a7 Avoid an infinite loop in fts3/4 incremental-merge in the case where the lowest level in the database contains segments but no data (because there is a delete-marker for each valid entry). Fix for [bf1aab89].
FossilOrigin-Name: 35beaee059a6cccead4311886ca928d936f23584cf435e35e265e98feea723dc
2019-10-17 15:41:36 +00:00
dan
8b4d0e2cba Ensure fts3/4 prefix indexes are flushed to disk before an 'optimize' command. Fix for [745f1abc].
FossilOrigin-Name: 4ed905b18847d4dbf5a2570052abced338065f824137b90300f62d77b3159d80
2019-10-11 15:33:13 +00:00
dan
9613c9f4a6 Fix the fts3/4 integrity-check command so that it works with "ORDER=DESC" tables. Fix for [8a6fa2bb].
FossilOrigin-Name: 5863546df99abd1ad4ebcaba18dec647be4368e2c13ce0b28e48c8608d3e87ef
2019-10-11 14:27:17 +00:00
dan
cd79010964 Fix a problem in the fts3 integrity-check routine causing it to report as corrupt tables that contain values with embedded 0x00 bytes.
FossilOrigin-Name: 629e20c9880acc2cb9e2fa6da39a15a90483e9a8f433df29d285c23bdfb03b0b
2019-10-10 16:41:44 +00:00
dan
23d3f5d661 Fix a long-standing problem in fts4 incrmental merge.
FossilOrigin-Name: 67da31e24ebb49c4cac81c9e7cfca37ca422555fd0fdb01d8f180890783c84ff
2019-10-02 19:33:34 +00:00
drh
5e81e9fdae The nodeReaderInit() function in FTS3 may not assume that the node is
non-empty.

FossilOrigin-Name: 361eb2f682a303bba72b39d322d9de630494ca044fe0508dcb23b6130d312d85
2019-09-30 19:13:31 +00:00
dan
8279fe8a4b Fix a crash in fts3 caused by corrupt database records.
FossilOrigin-Name: 11f7f94f1c5d761e7f381e277658b7e857cc369283996c509061de55fd6aad52
2019-07-29 16:53:30 +00:00
dan
7d840e530f Fix a valgrind problem in fts3corrupt4.test.
FossilOrigin-Name: cb3dec427e399064eeec31c15565346f045bd7c46d2f7860b1cd346bbcccb124
2019-07-05 15:16:22 +00:00
dan
471e88f732 Fix a segfault in fts3 that could occur when processing corrupt fts records.
FossilOrigin-Name: dff24c6c7a4e7623984b5bb24c2ff14fe079dc49a0ad048cbfc66f84c08875b0
2019-06-04 13:11:37 +00:00
dan
efdb371a1d Fix an assert() in fts3 that could fail when processing corrupt records. Also some minor sanitizer warnings.
FossilOrigin-Name: c736c40aab071a69bd5c5347b7c116d87ea3954019e967ca8ff7bdb4b518e5bf
2019-05-28 14:42:32 +00:00
dan
29fa6b6642 Fix a buffer overread that could in fts3 when dealing with corrupt records.
FossilOrigin-Name: 1660d7733eb443ab085ddef7666b998a1c75ac81cc54a9620960a8d3b377e28e
2019-05-22 14:22:44 +00:00
dan
bebea908b8 Fix another instance in fts3 where a corrupt record can cause a buffer overflow.
FossilOrigin-Name: e1724f1d618cfbcfd1e495d8965a395656cfc1114e1bffd4bc3be0bd5cdb6550
2019-04-24 16:13:52 +00:00
dan
04687f9e03 Fix an error in fts3_write.c allowing a corrupt database to cause a crash.
FossilOrigin-Name: 6e4a5f22811bcd1485e27073ec96821d448b91b6d1ff1659cc3772449d19a762
2019-04-24 15:13:02 +00:00
drh
3b574e4ea9 Use the 64-bit memory allocator interfaces in extensions, whenever possible.
FossilOrigin-Name: 07ee06fd390bfebebc014b47583d489747b0423bb96c810bed5c605ce0e3be71
2019-04-13 04:38:32 +00:00
drh
b43ac0021f Fix an assert() that might not be true if the %_segdir table of FTS3 contains
corrupt entries.

FossilOrigin-Name: a6509ca3dc2602c7d0376230ee63c1bb4c4066f007e77e9ac20d8eee966e5e13
2019-02-18 12:16:03 +00:00
dan
2fa619a3a8 Change a few assert() statements in fts3 that might fail if the database is corrupt.
FossilOrigin-Name: db74a56af73d92b7a9d43ceda7e4540915c580c68a0266b4ddefb9e0d5cbcbeb
2019-02-09 19:23:54 +00:00
dan
c1f6618217 Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt.
FossilOrigin-Name: 3498908cd7f3d0e35b70796537124e3da0bd99d48750ba51bcb9eba87e28ed4d
2019-01-25 14:48:18 +00:00
dan
2c7a73eaea Fix another segfault caused by a corrupt fts3 database.
FossilOrigin-Name: ba3b8412726548a0716c1a2d67260c3b7e31956474f4cd4ce607cf2cebc667dd
2019-01-22 12:21:28 +00:00
dan
4ccb41fc99 Fix a memory leak that could occur in fts3 when handling a corrupt database.
FossilOrigin-Name: 65cebb06a0afcbcb4157c3d518a62ed188b1e90d9e9b69d88fece484bcb6e380
2019-01-16 11:38:06 +00:00
dan
94ec92a1f7 Fix a problem with querying a corrupt fts3 database.
FossilOrigin-Name: 65e50706a017751d72d0f9c7f575955194977decd0acf13aaca90c3cb925f20e
2019-01-14 15:09:00 +00:00
drh
c49d43a787 Fix the fts3DecodeIntArray() function so that it will not read off the end
of the buffer it is handed.  Any unread integers are set to zero.

FossilOrigin-Name: 666cf8f6b39ae1f72e82b45e9cacba23caf61370ca0c695b3b14452accbb1a0d
2019-01-10 01:12:43 +00:00
dan
d8f3b4c320 Fix a problem with corrupt fts3 database detection introduced by [95a9a39f].
FossilOrigin-Name: c80bb15b44e9b6a3c0c5a49d5e93e66376c3520ecefc1db7425c8a7e09d3eade
2018-12-26 14:15:09 +00:00