Commit Graph

98 Commits

Author SHA1 Message Date
dan
f0f240c0d8 Add tests to restore coverage of fts5_storage.c.
FossilOrigin-Name: 38841881a08f4d75c2dfcbc1b2b45e311d7dda1287780f8ba1c8388f7adc1629
2024-08-16 17:54:00 +00:00
dan
e317e7f4b4 Change things so that locale=1 is required to write fts5_locale() values to an fts5 table, and so that blobs may not be stored in indexed (i.e. not UNINDEXED) columns of these tables.
FossilOrigin-Name: c98ccc12169419b8b27ead89ef0665de40320277c5daa748b80869337419e43e
2024-08-02 21:06:13 +00:00
dan
f971c15b1f Clarify the role of Fts5Storage.pSavedRow in the new feature on this branch.
FossilOrigin-Name: e8a61d5c48073fdd4d99d0b6fc70469b37af009f281336a44e3789e7eeed820d
2024-08-01 17:15:17 +00:00
dan
5bd8cc7dd5 Fix various problems with the code on this branch.
FossilOrigin-Name: 8bd4ae7e95c7b6ce34db5ea705dc136e742a22f333d0e7370b485ebd736b5ec2
2024-07-31 20:49:00 +00:00
dan
9015b8822d Fix cases surrounding blob values being stored in locale=1 fts5 tables.
FossilOrigin-Name: ae435aff1785d5832821c19dc88ccf6c496a7ff55be80276b31e5c9abee723db
2024-07-30 20:39:58 +00:00
dan
8f14c45197 Fix xColumnText() and xColumnSize() APIs. Add xColumnLocale() and xTokenizeSetLocale().
FossilOrigin-Name: 14006711d83d098e665c540b978b0e29aa8f458da1c2c8e9c2baf2ad5ebd502c
2024-07-29 20:31:17 +00:00
dan
53bef48fbe Ensure that an fts5 UPDATE does not strip the locale from columns that it does not write to.
FossilOrigin-Name: 69205264debd829573b1c777a5a493cfeb6083c4cdec106b1f819989f859ac75
2024-07-27 19:46:10 +00:00
dan
2ec78c0e4b Add the fts5_locale() function, and begin adding the related functionality to fts5.
FossilOrigin-Name: 8839ef7cfb49239e7f1c4812a53a93a672827c88d6921408b1d5062b352c87cc
2024-07-26 20:50:33 +00:00
drh
d109de8f84 Fix a problem in FTS5 where it reports the wrong error code following an OOM.
FossilOrigin-Name: 0c1cc4071edcd3e465779bbb17ea3ee6ddaa7c93c4fde55e5f3e07aa4b45783b
2024-07-18 18:44:44 +00:00
dan
e6289d6dd0 Improve the error message returned by an fts5 'rebuild' command on an external content table if there is a problem with the content table or view.
FossilOrigin-Name: 0fbf4b8a58fde1c187908934da6f59999b146f32e07ac255cc531c5c4d7007fd
2023-12-14 15:31:35 +00:00
dan
71bae9f19f Fix a problem allowing a COMMIT following an OOM to cause fts5 corruption.
FossilOrigin-Name: fba3129d1362dd3ee04d61cb3d3c48518cd98dde3febc85298f411776d60073e
2023-10-20 19:59:12 +00:00
dan
d1fbaa071b Fix various issues with code added to this branch.
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
2023-07-19 18:47:02 +00:00
dan
2cccced140 Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction.
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
2023-07-17 18:40:39 +00:00
dan
55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables.
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
2023-07-17 17:59:58 +00:00
dan
b184c91076 Have contentless_delete=1 tables support REPLACE statements.
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
2023-07-11 20:57:19 +00:00
dan
6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables.
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
2023-07-10 20:44:09 +00:00
drh
13ca72d011 Fix harmless compiler warnings, mostly caused by prior efforts to get static
analyzers to run without warnings.

FossilOrigin-Name: cddd3b382a118bb16be3a602ae711fe2c2a7d1c673dd30dd3977b67ee555e666
2021-10-05 18:59:05 +00:00
drh
d7ace22d8c Fix more harmless static-analyzer warnings. As of this check-in, no warnings
are generated by scan-build for clang-6 or clang-10.

FossilOrigin-Name: a7835bead85b1b18a8affd9835240b0baf9c7af887196bbdcc3f5d58055042fc
2021-10-05 18:33:38 +00:00
drh
7a3b4451a1 Fixes for harmless static-analyzer warnings. This also makes the code easier
for humans to understand.

FossilOrigin-Name: 36177a62feeb4fa93ab6e3c6f4dbe1ddcf63bb02f93284abab979da0261b218e
2021-10-05 17:41:12 +00:00
dan
7548ab20e6 In fts5 integrity checks, do not compare the contents of the index against an external content table unless specifically requested.
FossilOrigin-Name: 782163693f37aeb65209bebbaeb6659a36881b8c4b4bec778b366658488bf966
2020-09-21 14:53:21 +00:00
dan
86f477edaa Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems.
FossilOrigin-Name: b79f19edfd33c2a75f936c352668e14e81f35acf4f07edc27a21f941a7304b38
2020-09-11 15:01:49 +00:00
dan
42ae633ab9 Fix two more cases in fts5 where sqlite3_value_bytes() was being called before sqlite3_value_text(). Fix for e431c355.
FossilOrigin-Name: a1ba9a37d7a68a6d31f8197c6350589ebe6a12f4e3c193a178dd7ead8bcd565a
2019-12-20 19:41:01 +00:00
drh
b2ab3dcff0 Fix harmless compiler warnings.
FossilOrigin-Name: 2bbd014c900e0d71fd30ec647739b82c5dc3aa237a0203f5a1db50593dae199e
2019-12-20 14:08:34 +00:00
dan
73b21382d4 Ensure fts5 consistently calls sqlite3_value_text() before sqlite3_value_bytes(). This matters when storing blobs in fts5 tables within utf-16 databases. Fix for [cf36cb4e02].
FossilOrigin-Name: c16305eba0eb14368f2631b9a82b7eb4483c93f3b31257ee7c33f9b00c344379
2019-12-20 13:24:11 +00:00
dan
fbd79cc585 Fix problems with using the fts5 'rebuild' command inside a transaction that contains other updates of the same table. Fix for [e258f008].
FossilOrigin-Name: 238e0835714696aba0631f288fcc30ec5fddb43893d469c6bf017f386b3cddee
2019-10-08 13:34:24 +00:00
dan
3cbbd195ca Prevent an fts5 table from being its own content table, or part of a view that is the content table.
FossilOrigin-Name: b6d52c9364767ff4ab7279ae981afb97799299dcfaf38a0110c40ca82c72a825
2019-08-05 12:55:56 +00:00
mistachkin
065f3bf4f2 Fix various harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c
2019-03-20 05:45:03 +00:00
dan
27a037354c Avoid a division-by-zero error in fts5 caused by a corrupt database.
FossilOrigin-Name: 928e622178bd004be52b2271ac554176aebe9d6e115ecac4e7072eccbb4262b5
2019-01-19 15:55:27 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
2019-01-08 20:02:48 +00:00
dan
c210c08891 Fix two more problems with corrupt database handling in fts5.
FossilOrigin-Name: 444c7c99beb9f8b82963e9784aa2be54124c7011f4771122b4f608c02aa26408
2019-01-05 07:17:56 +00:00
dan
1ea0443c2d Add new sqlite3_prepare_v3() flag SQLITE_PREPARE_NO_VTAB, for preparing
statements that are not allowed to use any virtual tables. Use this to prevent
circular references in triggers on virtual table shadow tables from causing
resource leaks.

FossilOrigin-Name: 25666e3d03950caf753295cdb55df162e07dbcf6840b05875c6e0b127c469ecb
2018-12-21 19:29:11 +00:00
dan
eab0e10304 In extensions rtree, fts3 and fts5, ensure that when dynamic buffers are bound
to persistent SQL statements using SQLITE_STATIC, the binding is replaced with
an SQL NULL before the buffer is freed. Otherwise, a user may obtain a pointer
to the persistent statement using sqlite3_next_stmt() and attempt to access
the freed buffer using sqlite3_expanded_sql() or similar.

FossilOrigin-Name: 2a5f813bc61f9e780f2ccbda425611f65ad523b6d486a1e5e2b9d5e9f1d260a2
2018-02-07 18:02:50 +00:00
drh
2c2f392dca Add interfaces sqlite3_prepare_v3() and sqlite3_prepare16_v3() with the
extra prepFlags argument.  Add the SQLITE_PREPARE_PERSISTENT option as one
bit in that argument.

FossilOrigin-Name: 4a25c5883380fe5990d8180adb58c3bdc7a3d081bc4c69cd4de3cd57074fb251
2017-06-01 00:54:35 +00:00
dan
be0bc8bb4d Have fts5 close any open blob-handle when a new savepoint is opened. This
ensures that fts5 does not prevent DROP TABLE statements (which always open a
savepoint) from succeeding.

FossilOrigin-Name: a921ada89050ce1d162fd1b0056939573635e2cec7ac0c2a99ae924b3ae593f7
2017-04-08 09:12:20 +00:00
dan
81415257ec Improve performance of fts5 writes by writing the "averages" record once at
the end of each transaction instead of every time the table is updated.

FossilOrigin-Name: 2b210d691462b463796e7948399133c296e89ebf
2017-02-27 17:06:56 +00:00
dan
9c58b63c18 Add an sqlite3_set_last_insert_rowid() method. Use it to work around fts4 and
fts5 modifying the last-insert-rowid unintuitively from within commit
processing.

FossilOrigin-Name: fe41bb5632a5d438acfd682809f1bd12315b970a
2017-02-27 14:52:48 +00:00
dan
8414e81359 Add the largely untested SQLITE_FTS5_NO_WITHOUT_ROWID compile time option to fts5. For building a dynamically loadable extension that does not use WITHOUT ROWID.
FossilOrigin-Name: d0a1cf1c56c237617fb73cb31f4950365b1f3e9b
2016-07-15 19:17:19 +00:00
dan
848b190e40 Explicitly limit the size of fts5 tokens to 32768 bytes.
FossilOrigin-Name: 70fc69eed9b09159899d7cbd1416a59d04210a63
2016-03-23 15:04:00 +00:00
dan
f5268268b7 Have fts5 cache the structure of its index in main memory. Use "PRAGMA data_version" to figure out when this cache should be invalidated.
FossilOrigin-Name: 902f221754f3cc6fb4ae049c135f85efce604ed3
2016-03-21 15:30:50 +00:00
drh
860443da10 Fix FTS5 so that it works with SQLITE_OMIT_AUTORESET.
FossilOrigin-Name: b199637d81d7e2a767131ac03c7679b101fd459c
2016-03-18 00:39:40 +00:00
dan
f55fb6615b Have fts5 cache the decoded structure of fts5 indexes in memory. Use "PRAGMA data_version" to detect stale caches.
FossilOrigin-Name: 33ef2210ef19e55c8d460bfe9d3dc146034c8acc
2016-03-16 19:48:10 +00:00
dan
22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code.
FossilOrigin-Name: bc3f7900d5a06829d123814a5ac7b951bcfc1560
2016-02-11 17:01:32 +00:00
dan
e9eb1593f5 Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode.
FossilOrigin-Name: 72d53699bf0dcdb9d2a22e229989d7435f061399
2016-01-23 18:51:59 +00:00
dan
25247435be Rationalize some code in fts5_storage.c. Add tests to cover recently added branches in fts5.
FossilOrigin-Name: 3b72df405ac9b3a71144f45317d32e25f0084c4b
2016-01-15 19:54:47 +00:00
dan
4d14398a61 Merge latest trunk changes into this branch.
FossilOrigin-Name: e7dcd013516723941c2fb78ecdc96806d49d310b
2016-01-06 19:35:52 +00:00
dan
811501e237 Fix the fts5 integrity-check code so that it works with detail=none tables.
FossilOrigin-Name: 3a1df69e58e5830da1dff158eedbe3817743f58f
2015-12-31 18:39:14 +00:00
dan
9f44deed93 Change the name of the offsets=0 option to "detail=column". Have the xInst, xPhraseFirst and other API functions work by parsing the original text for detail=column tables.
FossilOrigin-Name: 228b4d10e38f7d70e1b008c3c9b4a1ae3e32e30d
2015-12-28 19:55:00 +00:00
dan
3e6a141130 Fix some harmless gcc compiler warnings. Mostly in fts5, but also two in the core code.
FossilOrigin-Name: 5d44d4a6cf5c6b983cbd846d9bc34251df8f4bc5
2015-12-23 16:42:27 +00:00
dan
159fd77e0f Fix an fts5 integrity-check problem that affects offsets=0 tables with prefix indexes.
FossilOrigin-Name: 609a0bc7f34e6dae74ce756aff920f3df78fe828
2015-12-21 18:45:09 +00:00
dan
b12dc84fbb Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch.
FossilOrigin-Name: 40b5bbf02a824ca73b33aa4ae1c7d5f65b7cda10
2015-12-17 20:36:13 +00:00