mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
28,421 Commits 1,487 Branches 166 Tags 416 MiB
342984075b
Commit Graph

169 Commits

Author SHA1 Message Date
dan
10e54e365a Fix further issues to do with fts5 locale support. 
FossilOrigin-Name: e626123580065986f7df50b6140f00048944becce179b9391fbf09f97ac55485
 2024-08-12 17:03:37 +00:00
dan
9b118de9d5 Merge trunk changes into this branch. 
FossilOrigin-Name: b243007525a825b3daf8aa9bb2d3088efb853bd8b7c9ea3c0924fde193eb5c44
 2024-08-10 20:03:01 +00:00
dan
ff6ab9dd2b Update the fts5_tokenizer_v2 API so that the locale is passed as parameter to xTokenize(), instead of via a separate call to xSetLocale(). 
FossilOrigin-Name: 03e63ed24e7a08817341e59b075ae2d4e3f7a5e5b37e0e6138359d5fd29a5e9e
 2024-08-10 18:59:36 +00:00
dan
679f5d458e Add assert() statements and reorganize code slightly in fts3 and fts5 to make it easier to follow. 
FossilOrigin-Name: 797b0a13fd7a42b0a48ecbf0cd1961aa932da3e9c9ccffd903a3a4d963d0cc54
 2024-08-08 15:07:27 +00:00
dan
5cd01c30b4 Fix a broken assert in fts5_expr.c. 
FossilOrigin-Name: 08cc5488404d068e59378b82988460793710df43ec21b4a83a794b497abd035f
 2024-08-07 21:20:08 +00:00
drh
3d24637325 Add assert() statements to FTS5 to hush-up warnings from scan-build. 
FossilOrigin-Name: 77a76654e64c05f2c67be120f579fc60924ff51a9d0fa2cf9eb158f40171b5f5
 2024-07-02 13:54:46 +00:00
dan
fdda1cbe0c Fix some problems with calling fts5 api functions from within non-full-text queries. 
FossilOrigin-Name: 56d265f956fe6433b625c6d732e55f387af3f643e705316f5a6f709d63731669
 2024-06-24 16:08:01 +00:00
dan
0e7a7e57cc Fix a crash in fts5 that could occur within "PRAGMA integrity_check" if the specified tokenizer was unknown. 
FossilOrigin-Name: 061d7b69e6a43474cd7123a69f4ee65b5649a0b5e51474343defef5ae5262657
 2024-05-24 21:23:36 +00:00
dan
32ca0dbcdf Have fts5 tables delay initializing the tokenizer until it is first used in all cases where the tokenizer is not "trigram". 
FossilOrigin-Name: ca4fdcb8ae95d2a61236b949f852d2bf25ea2dbbff7eedafbd8eb84e8fd96687
 2024-05-14 17:16:09 +00:00
drh
42203e1b75 Fix harmless compiler warnings in FTS5. 
FossilOrigin-Name: 3cd5ef44e40570c357f913a9483fa1cd72e7f2827a5ed5826bff99febae213b1
 2023-12-28 16:25:48 +00:00
dan
d82320ac9a Ensure that the xColumnText(), xQueryPhrase() and xPhraseFirstColumn() APIs all return SQLITE_RANGE if they are passed a bad column or phrase number. 
FossilOrigin-Name: 1a8a9b1c89519d265869251e8b6d3c5db733f0d3a7dea6c7962811a8f1157dff
 2023-12-26 15:52:40 +00:00
drh
4a5c96ace3 Fix harmless compiler warnings in JSON and FTS5. 
FossilOrigin-Name: 90135efccfeb1046f002bfcbd8dfec9a1a3b40cbe1b5c714ae065b06368e354f
 2023-12-14 15:38:57 +00:00
dan
fb923fc4cc Further tests for the new code on this branch. 
FossilOrigin-Name: 59d008b6c23ab900377bc696ee19381feb7614bac80546eae361e401c3620c4e
 2023-12-05 18:36:23 +00:00
dan
54318b382a Fixes for xInstToken() with tokendata=0 tables. And with prefix queries. 
FossilOrigin-Name: 78fbb71598b1ca756acc078253880a1d0f7983a5a26b9efc683e6488122505a1
 2023-12-04 18:45:14 +00:00
dan
3dfc063705 Fix a problem with the xInstCount() API and "ORDER BY rank" queries. 
FossilOrigin-Name: 317a50563d9e8586fda136e513727241b414e7267d50a06571c8ebd0eae710bc
 2023-12-04 17:45:33 +00:00
dan
c22d2b7b7f Fix various compiler warnings and other problems with the new code on this branch. 
FossilOrigin-Name: 3a623cfa173b4035c759cb84985d11d8727053beb383648503987d6ab15c0ef0
 2023-12-02 20:35:04 +00:00
dan
c44041e03b Ensure that tokendata=1 queries avoid loading large doclists for queries like "common AND uncommon", just as tokendata=0 queries do. 
FossilOrigin-Name: 7bda09ab404a110d57449e149a3281fca8dc4cacf7bd9832ea2a1356ad20fe8e
 2023-12-02 17:32:16 +00:00
dan
b5effc0605 Different approach to querying a tokendata=1 table. Saves cpu and memory. 
FossilOrigin-Name: c523f40895866e6fc979a26483dbea8206126b4bbdf4b73b77263c09e13c855e
 2023-12-01 20:09:59 +00:00
dan
af54826e4a Defer building xInstToken() hash-table until it is to be used. 
FossilOrigin-Name: 9b005085ff4a53cda0a1dff0c836630d6d3b95b9c40658ffd2a886f3e1b37faa
 2023-11-22 20:02:55 +00:00
dan
5c268bbf67 Fix tokendata=1 and xInstToken() APIs for detail=none and detail=column tables. 
FossilOrigin-Name: 37b271c19d772bd06524db816ded03377b426efed7a7783c8a96f6fb156ecd86
 2023-11-22 19:02:54 +00:00
dan
50b0e25a55 Add implementation of xInstToken() API. 
FossilOrigin-Name: a34b26fe7f60b74e7ae5cf64900920a3d352a20da2496401bcbc27041689cd07
 2023-11-15 11:45:19 +00:00
dan
e108029332 Add new fts5 API xQueryToken(). 
FossilOrigin-Name: 828566392b3ea8db603cb1ae5eccbc8ac035efaa284bc7c15ba89874f634aec9
 2023-11-13 14:29:12 +00:00
dan
03204e9106 Add the tokendata=1 option to ignore trailing token-data when querying an fts5 table. 
FossilOrigin-Name: 122935182ad5869ce3a4c6d796c38a0509f6f3384dd1b3e60a3f2f0f366cc5f5
 2023-10-11 21:08:12 +00:00
dan
1846de49a4 Fixes for fts5 expression parser module to allow embedded 0x00 bytes in tokens. 
FossilOrigin-Name: 342c8d0783f449817d3f565ff6b9f010a6c690beeea32f1861640810490a8b5f
 2023-10-03 19:06:52 +00:00
dan
8c5611a169 Fix problems in fts5 test code. 
FossilOrigin-Name: c16ed3b1cc7b4c21bfa96062bf9028ae071c89e63df3b444b59f1753594e4e94
 2023-08-15 14:53:49 +00:00
drh
9faa3bbdbb Get access to SQL functions and virtual tables used for debugging and analysis 
of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST.

FossilOrigin-Name: 383de8e2259adb3f8881a618b8b5e06c1f3b2bd7ac274bd047410ad7d2a18f0f
 2023-07-12 18:38:47 +00:00
dan
4042d2b424 Limit the number of nested NOT nodes in an fts5 expression to 256. 
FossilOrigin-Name: 0e5c1ee40a146ef8b2b3c5f53d0a45e092bc8d8e933f3819805c995819d31bae
 2023-05-15 17:14:16 +00:00
dan
4d24eff8dc Avoid a crash in fts5 when combining an empty expression with another. 
FossilOrigin-Name: 33681ff3d29c9f096dbd8c5d885993f780e93974a7653adc00fa41a158356d7c
 2023-04-22 20:47:08 +00:00
dan
a3e6192941 Fix a problem with the fts5 trigram tokenizer and LIKE or GLOB patterns for which contain runs of 2 or fewer non-wildcard characters that are 3 or more bytes when encoded as utf-8. 
FossilOrigin-Name: 00714b39b39c51519edbc0194f98c7275fecf96763a06fd95db6e1d81bb9f1f1
 2023-02-10 17:17:04 +00:00
drh
c381056e6d Fix harmless compiler warnings. 
FossilOrigin-Name: 61095b5bcb5c30d29f03efa5b4809e3a0b780ed57e35227c292aab9dd1cfe3f6
 2022-04-07 10:11:35 +00:00
dan
76c3ae6848 Fix a broken assert() in fts5 that could fail if the snippet() function were used with a query phrase containing 0 tokens on a detail=none or detail=col table. 
FossilOrigin-Name: 0a5b94b99cf45df759a59cb5fc62da111ca33f8c2614769a81930d72b51f093a
 2021-11-02 07:32:13 +00:00
drh
7a3b4451a1 Fixes for harmless static-analyzer warnings. This also makes the code easier 
for humans to understand.

FossilOrigin-Name: 36177a62feeb4fa93ab6e3c6f4dbe1ddcf63bb02f93284abab979da0261b218e
 2021-10-05 17:41:12 +00:00
drh
11a9ad5669 Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. 
Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause
ALWAYS() and NEVER() macros to be omitted from the build.

FossilOrigin-Name: 1c67f957fc77e37ce8f0d447c41ca975e8e79a35d332739c24a633649b5b0387
 2021-10-04 18:21:14 +00:00
dan
90d3b24a16 Fix a memory leak in fts5 that could occur when processing a query containing multiple classes of error. 
FossilOrigin-Name: 7c279670f544a5bb9733a88862d9e490062b07415cd790e0d6d1e2cb584d2007
 2021-06-30 14:04:25 +00:00
drh
214b50314a Fix harmless compiler warnings of unused function parameters in FTS5. 
FossilOrigin-Name: 14273915979f8fbf80111e795b5511a6ee24638896b9d434d6ccf4bf7f7a7f62
 2021-04-22 19:34:31 +00:00
dan
6a4f7f0225 Remove fts5 test functions fts5_expr(), fts5_expr_tcl(), fts5_isalnum() and fts5_fold() from release builds. 
FossilOrigin-Name: c68a6c6c898d2acd1f7032ff98f8f3f99eb0b37ffbc6ee88d0dd7badf55a0e0a
 2021-04-19 16:45:09 +00:00
dan
84b168ffa7 Change an fts5 assert() that can be triggered by a corrupt database to an if() condition. 
FossilOrigin-Name: ea0a7f103a6f6a9e57d7377140ff9f372bf2b156f86f148291fb05a7030f2b36
 2020-12-15 13:55:38 +00:00
dan
f46be6a1b9 Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE. 
FossilOrigin-Name: 64782463be62b72b5cd0bfaa7c9b69aa487d807c5fe0e65a272080b7739fd21b
 2020-10-05 16:41:56 +00:00
dan
ccf578d435 Add tests for the trigram tokenizer. Fix minor issues. 
FossilOrigin-Name: 897ced99b44085012aa44d3264940dcbd4c77b295a894a1b58fb2c03a0f7fee8
 2020-10-01 16:10:22 +00:00
dan
33a99fad08 Add experimental unicode-aware trigram tokenizer to fts5. And support for LIKE and GLOB optimizations for fts5 tables that use said tokenizer. 
FossilOrigin-Name: 0d7810c1aea93c0a3da1ccc4911dbce8a1b6e1dbfe1ab7e800289a0c783b5985
 2020-09-30 20:35:37 +00:00
dan
86f477edaa Catch fts5 index corruption caused by issuing 'delete' commands with incorrect data earlier in some cases. Also fix a couple of test script problems. 
FossilOrigin-Name: b79f19edfd33c2a75f936c352668e14e81f35acf4f07edc27a21f941a7304b38
 2020-09-11 15:01:49 +00:00
dan
832aa023c6 Add fts5 test to confirm that for a table with columns a, b, c and d, "{a b}" and "-{c d}" are handled similarly. 
FossilOrigin-Name: 1a04920998368e56276fd0b100be8343609c6ff8a731cf8e26a0490f9c6dabdf
 2020-08-28 11:19:49 +00:00
dan
c19e22f397 Fix possible null pointer dereferences in the fts5_expr() scalar function. 
FossilOrigin-Name: c5d44143599f3fe98492b2b900fa3d77925c7be545096251055ceeab899a41f1
 2019-12-09 02:20:37 +00:00
dan
685b2ee0c3 Allow fts5 to filter on multiple MATCH clauses in a single scan. 
FossilOrigin-Name: 9d418a7a491761eeb38a70898677a493e2631e5d62e75ee88431f52d3dfd2344
 2019-09-12 19:38:40 +00:00
mistachkin
065f3bf4f2 Fix various harmless compiler warnings seen with MSVC. 
FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c
 2019-03-20 05:45:03 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions. 
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
 2019-01-08 20:02:48 +00:00
dan
b163b57212 Fix problems in fts5 found by ASAN. 
FossilOrigin-Name: c564bf870106faef297594a51995619c80311d06bd5f8a0c7644f666f22ba576
 2018-12-28 07:37:22 +00:00
dan
b80bb6ce88 Add the "categories" option to the unicode61 tokenizer in fts5. 
FossilOrigin-Name: 80d2b9e635e3100f90cffdcffa5b5038da6fbbfccc9f5777c59a4ae760d4cb62
 2018-07-13 19:52:43 +00:00
dan
d2ac43ea38 Avoid a warning in fts5.c due to not using generated routine 
sqlite3Fts5ParserFallback().

FossilOrigin-Name: 65ff5144a55f1ba2b2ef812a0c988fa2522441177038b8d686eb1d0c558c75d5
 2018-06-30 20:26:52 +00:00
mistachkin
23355215fb Fix harmless compiler warning in FTS5. 
FossilOrigin-Name: 2d544711c7e9f4c2c3ca7ee7d588948d048da82e63c82f41c999fce947d6661d
 2018-03-07 14:49:32 +00:00