Commit Graph

842 Commits

Author SHA1 Message Date
dan
2ec78c0e4b Add the fts5_locale() function, and begin adding the related functionality to fts5.
FossilOrigin-Name: 8839ef7cfb49239e7f1c4812a53a93a672827c88d6921408b1d5062b352c87cc
2024-07-26 20:50:33 +00:00
drh
d109de8f84 Fix a problem in FTS5 where it reports the wrong error code following an OOM.
FossilOrigin-Name: 0c1cc4071edcd3e465779bbb17ea3ee6ddaa7c93c4fde55e5f3e07aa4b45783b
2024-07-18 18:44:44 +00:00
dan
a592883d87 Add extra assert() statements to the fixes on this branch.
FossilOrigin-Name: 81b6360050eea95e4367de1b41b5864a640b4d1c5c8fc3bea3b96ed770cb0325
2024-07-17 16:27:36 +00:00
dan
7acf972c59 Fixes for platforms with 32-bit pointers that require 64-bit values to be aligned.
FossilOrigin-Name: 2212d7488ed4ec2839ffa45cb9567056b36519434834634e4ecc441c330694d7
2024-07-13 16:53:56 +00:00
dan
bee10b987c Improve the error message returned when fts5 finds a row is missing from its content table.
FossilOrigin-Name: 9b1afd4acb8cf9738154e921dcb074d895bbcc4742bf1016ef9f045a0da78bc8
2024-07-08 16:01:49 +00:00
drh
3d24637325 Add assert() statements to FTS5 to hush-up warnings from scan-build.
FossilOrigin-Name: 77a76654e64c05f2c67be120f579fc60924ff51a9d0fa2cf9eb158f40171b5f5
2024-07-02 13:54:46 +00:00
dan
75bdf00e89 Update fts5 comment/docs to include the order matches are visited in by the xPhraseFirst/xPhraseNext API.
FossilOrigin-Name: e41930ef5b883c0f2a6b7eb9606021d14a3df145bc94cb0d508af22b4db7f168
2024-06-28 18:21:30 +00:00
dan
105c20648e Fix an incorrect tcl comment that appeared in many fts5 test files.
FossilOrigin-Name: d07085e2035b52a7edd27980523225e59c5bf851fb4a6de975f03e653b937c9c
2024-06-24 18:06:15 +00:00
dan
fdda1cbe0c Fix some problems with calling fts5 api functions from within non-full-text queries.
FossilOrigin-Name: 56d265f956fe6433b625c6d732e55f387af3f643e705316f5a6f709d63731669
2024-06-24 16:08:01 +00:00
mistachkin
6593b340ff Fix harmless compilation issues seen with MSVC.
FossilOrigin-Name: 816d4749384c7f398912c905a16c83b88f4c55632050b4c6117c61301d1a53e1
2024-06-05 20:50:39 +00:00
dan
6d98c55b56 Fix an fts5 problem with secure-delete mode causing integrity-check to erroneously report a corrupt index.
FossilOrigin-Name: 80bef4d60ba9e3679ea66655ca36fcfaa888775a3d1598d50e9649ad84a95b63
2024-06-05 14:47:54 +00:00
dan
9c3eef8c0d Have fts5.xBestIndex return SQLITE_CONSTRAINT, instead of a large cost, when no usable plan can be found.
FossilOrigin-Name: 7c470945372dc98610f7c9840ce8cab18c19b655352f0187e4f31040cea77363
2024-06-01 17:56:58 +00:00
drh
7da33383c7 Remove an unused parameter from fts5ConfigParseSpecial(). Compiler-warning
fix only - no functional changes.

FossilOrigin-Name: c08dd245f7706f2fd2269d700be480477619a722e27e6b439462ae543302c49f
2024-05-29 15:16:17 +00:00
dan
2386943f19 Fix a problem with the error message reported by fts5 in some OOM cases.
FossilOrigin-Name: c7aaa28f786567e29ebee3848283b5c740dd3b8144dc14372e56260ae1b76922
2024-05-28 10:57:24 +00:00
dan
c598fdd10c Fix another fts5 problem related to unknown tokenizer modules.
FossilOrigin-Name: e78b7783040ef1ad34d4287af8a7a5998d360d50076e0de42a116d9a96fe74b4
2024-05-24 21:38:37 +00:00
dan
0e7a7e57cc Fix a crash in fts5 that could occur within "PRAGMA integrity_check" if the specified tokenizer was unknown.
FossilOrigin-Name: 061d7b69e6a43474cd7123a69f4ee65b5649a0b5e51474343defef5ae5262657
2024-05-24 21:23:36 +00:00
dan
a0693f21fc Optimize queries against fts5vocab(row) tables that do use the "cnt" column.
FossilOrigin-Name: 96a591c202cb4218999ece3de063bf466b6fd81bd23383893f3913ecb52dda36
2024-05-14 19:41:19 +00:00
dan
433af7ab93 Avoid calculating the value of the "cnt" column for fts5vocab(row) queries that do not use it.
FossilOrigin-Name: c0c91e306c252ce1b398e92ab958e2a3de0fc43db05cfdffc548380654fc9f13
2024-05-14 19:09:54 +00:00
dan
32ca0dbcdf Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram".
FossilOrigin-Name: ca4fdcb8ae95d2a61236b949f852d2bf25ea2dbbff7eedafbd8eb84e8fd96687
2024-05-14 17:16:09 +00:00
dan
12b205c637 Allow existing fts5 tables to be dropped even if the associated tokenizer is not available.
FossilOrigin-Name: 69ef47eeee8b53684c321393be34f03600694fbc86377f8720ff80307846aff6
2024-05-13 20:06:08 +00:00
dan
6921a2d205 Update comments in fts5.h.
FossilOrigin-Name: 862945d5c432c27377e90d93d64c4655eefcc369d086eb51edef925fb3d80b57
2024-03-27 20:34:14 +00:00
dan
5e5831a760 Handle an OOM case in fts5 code to query tokendata=1 tables.
FossilOrigin-Name: 01222d96b9ab24c7013a2587544304ba32c1f039b08c06a04d8d58ec9c790db0
2024-02-16 11:41:47 +00:00
mistachkin
3ab9c021ff Fix harmless compiler warnings seen with MSVC.
FossilOrigin-Name: e52c87420b072fa68d921eda66069542d50accbfaf1110ac4cc1543a4162200d
2024-02-04 04:01:11 +00:00
dan
09768559b6 Ensure the fts5 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
FossilOrigin-Name: d18d9a05c2d6f290d7b16eafced0cde4eb8dd3f869778a49103812d35a0cee65
2024-02-01 15:42:22 +00:00
dan
75f9ba32bd Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases.
FossilOrigin-Name: b855886c4ccce0745af6957943e77be18949722f09821688725d546d3d79b4fb
2024-01-16 16:05:21 +00:00
dan
c08a8a33d3 In fts5, flush the contents of the in-memory hash table whenever the secure-delete option is toggled. This prevents spurious corruption reports under some circumstances.
FossilOrigin-Name: ccf552319a62bfb329820a3bc1f490bacbaa6e90694a257fc65a568a605542c3
2024-01-11 14:03:08 +00:00
dan
4eb626350a Fix a problem in fts5 caused by a COMMIT involving fts5 data that immediately follows a ROLLBACK TO that does not.
FossilOrigin-Name: 55c61f6a8d6a1bc79497b05669beac5c5397b06382bf24b6bec54845962d219b
2024-01-02 21:08:25 +00:00
dan
751c4e2152 Update fts5origintext4.test to work with SQLITE_DIRECT_OVERFLOW_READ.
FossilOrigin-Name: 15ed002aed12556aeb9bbe537c4ba839f0c95bac65a69d03401b37cc3fd11b92
2023-12-28 20:54:12 +00:00
drh
42203e1b75 Fix harmless compiler warnings in FTS5.
FossilOrigin-Name: 3cd5ef44e40570c357f913a9483fa1cd72e7f2827a5ed5826bff99febae213b1
2023-12-28 16:25:48 +00:00
dan
d82320ac9a Ensure that the xColumnText(), xQueryPhrase() and xPhraseFirstColumn() APIs all return SQLITE_RANGE if they are passed a bad column or phrase number.
FossilOrigin-Name: 1a8a9b1c89519d265869251e8b6d3c5db733f0d3a7dea6c7962811a8f1157dff
2023-12-26 15:52:40 +00:00
drh
c1805ab222 Avoid signed integer overflow during integrity_check of FTS5.
FossilOrigin-Name: 5937df3b25799eceaadfb04d7226c9995d44c8d8edb5ac3ad02af9d7e3570726
2023-12-24 11:31:20 +00:00
dan
0810150532 Fix a usan complaint about signed integer overflow.
FossilOrigin-Name: e65907e0279f4814ec957f0790777d8b94a86926cd27c52442b311b27efc0185
2023-12-22 14:47:30 +00:00
dan
f2194605f8 Do not run test script fts5origintest4.test with either "memsubsys1" or "mmap" permutations.
FossilOrigin-Name: 05a63d9603ef42cbee6dadff72d97583a9c78e549f70e9a808534d5c1ae7c28a
2023-12-15 15:17:39 +00:00
drh
eb78d56ec5 Add assert()s to FTS5 to fix static analyzer warnings.
FossilOrigin-Name: 27d4a89a5ff96b7b7fc5dc9650e1269f7c7edf91de9b9aafce40be9ecc8b95e9
2023-12-14 16:34:47 +00:00
drh
4a5c96ace3 Fix harmless compiler warnings in JSON and FTS5.
FossilOrigin-Name: 90135efccfeb1046f002bfcbd8dfec9a1a3b40cbe1b5c714ae065b06368e354f
2023-12-14 15:38:57 +00:00
dan
e6289d6dd0 Improve the error message returned by an fts5 'rebuild' command on an external content table if there is a problem with the content table or view.
FossilOrigin-Name: 0fbf4b8a58fde1c187908934da6f59999b146f32e07ac255cc531c5c4d7007fd
2023-12-14 15:31:35 +00:00
dan
05980f5931 Avoid dropping an error code in new fts5 tokendata=1 code.
FossilOrigin-Name: a66596e33dc9aa4bab2ec3ff45546e1321d0a11bdc764f8381b315292ca92423
2023-12-08 12:04:32 +00:00
dan
0b427eb796 Ensure an fts5vocab table never uses a special tokendata=1 merge cursor.
FossilOrigin-Name: 1e26510e83b40c9bd2e8bfa2a0e81f2cb915e78fed773204ef537683e48b61dc
2023-12-07 21:09:33 +00:00
dan
7c12666fe9 Avoid an assert() failure when querying an fts5vocab table that accesses a tokendata=1 fts5 table with corrupt %_data records.
FossilOrigin-Name: 386ba9e20423fb2f623d6adc9d3c310fb1b135f54a1dad15ef3b593d97886926
2023-12-07 20:46:40 +00:00
dan
1ddcf7dd95 Fix a null-pointer dereference in fts5 tokendata=1 code.
FossilOrigin-Name: d69fa8f0504887f968d9a190ecb889ddb40bb1b56d0d4479f9819c106aec719b
2023-12-07 19:08:25 +00:00
dan
5f2f8d0028 Fix a problem with handling OOM and other errors in fts5 when querying tokendata=1 tables.
FossilOrigin-Name: bc911ab5953532956510c199be72b1d3c556f2d0ddbd7fc0ae6f5f917b337b48
2023-12-07 18:41:49 +00:00
dan
2f3ccb8886 Different fix for the fts5 COMMIT-following-OOM problem first fixed by [fba3129d]. This one does not cause problems if an fts5 table is renamed and then dropped within the same transaction.
FossilOrigin-Name: d8c6b246944934a7a6e027b3f5b986fd64a19dd5c5c5175f4ea8586da59a6764
2023-12-07 14:41:58 +00:00
dan
00f3ac5544 Fix compiler warning about shadowed variable in fts5_index.c.
FossilOrigin-Name: ee70e4c1c9c41617850228e48d8df44f105cf2fbbe789340ceca6f27ad6ce5eb
2023-12-06 18:34:59 +00:00
dan
83ac79282a Update documentation comments in fts5.h.
FossilOrigin-Name: 38c50e22c98607e6c1fd78d7615cda534773b6d4fd85c712b54749fcd7af0c83
2023-12-06 18:10:50 +00:00
dan
fb923fc4cc Further tests for the new code on this branch.
FossilOrigin-Name: 59d008b6c23ab900377bc696ee19381feb7614bac80546eae361e401c3620c4e
2023-12-05 18:36:23 +00:00
dan
49bfbc1ef3 Add further tests for xInstToken().
FossilOrigin-Name: 8582707f16133f003a6687f68cbea03d4eb6c2a0e2e07746b7cace0c44e84fa4
2023-12-04 19:48:08 +00:00
dan
54318b382a Fixes for xInstToken() with tokendata=0 tables. And with prefix queries.
FossilOrigin-Name: 78fbb71598b1ca756acc078253880a1d0f7983a5a26b9efc683e6488122505a1
2023-12-04 18:45:14 +00:00
dan
910c77b049 Fix memory leak in new code on this branch.
FossilOrigin-Name: ebc160b9a05568df66f86e30804399ee29d34b44a60c57e062f98cb92826353f
2023-12-04 17:58:56 +00:00
dan
3dfc063705 Fix a problem with the xInstCount() API and "ORDER BY rank" queries.
FossilOrigin-Name: 317a50563d9e8586fda136e513727241b414e7267d50a06571c8ebd0eae710bc
2023-12-04 17:45:33 +00:00
dan
9d373ca1c5 Fix bug in xInstToken() causing the wrong token to be returned.
FossilOrigin-Name: da78d07e77cbc783fbc725758911c230fd6a1c1885d9576125de955dcc2bd37f
2023-12-04 17:05:37 +00:00