Commit Graph

50 Commits

Author SHA1 Message Date
dan
cad760d16e Fix a compiler warning in fts5_aux.c.
FossilOrigin-Name: 8edb983bc87898eff2cd2e7e672a32a47c71b2be9d818513d339e95560d45b2b
2020-11-27 19:40:13 +00:00
dan
6d19bf9460 Add missing cast to fts5 bm25() code.
FossilOrigin-Name: 6ff9673847c0b4174d9435e93d19af0ee7406b1a12edeb6edec98697e1646824
2020-11-27 16:15:55 +00:00
dan
3181706895 Remove a redundant "if( rc==SQLITE_OK )" from the fts5 bm25() code.
FossilOrigin-Name: d85f4f27f58adcc75fc7d59e63af95b2a338052d8748a11f22ec1e48d1aff4cc
2020-11-27 16:08:15 +00:00
dan
66efc393c5 Move an "if( rc==SQLITE_OK )" to outside a loop body in the fts5 bm25() code.
FossilOrigin-Name: 14a4dcf3474566d072007a37d214c892397c21dd3b7f8b55ad0e5edfb7130dd6
2020-11-27 16:05:31 +00:00
mistachkin
065f3bf4f2 Fix various harmless compiler warnings seen with MSVC.
FossilOrigin-Name: 1c0fe5b5763fe5cbace9773dcdab742e126d0bd035ab13d61f9d134afa0afc0c
2019-03-20 05:45:03 +00:00
dan
e25646d64c Avoid an integer overflow in the fts5 snippet() function triggered by a corrupt database record.
FossilOrigin-Name: 7c862c4655841e1263cf548d01f5b4331685ab7d396494891cad0a5f50c63e16
2019-01-29 16:34:45 +00:00
dan
e88609f23e Fix asan warnings in fts5 triggered by corrupt databases - passing NULL to memcmp, out-of-range left-shift values and signed integer overflow.
FossilOrigin-Name: 93f8ec146d63af13f04e337ada4fa75e9254f72b1394df09701ae12e185f27e2
2019-01-25 16:54:06 +00:00
dan
27a037354c Avoid a division-by-zero error in fts5 caused by a corrupt database.
FossilOrigin-Name: 928e622178bd004be52b2271ac554176aebe9d6e115ecac4e7072eccbb4262b5
2019-01-19 15:55:27 +00:00
dan
1a39d2bb5c Handle SQL NULL values without crashing in the fts5 snippet() and highlight()
functions.

FossilOrigin-Name: a5e9cc794f17566160c4b2b5791601a744a21ace987b19ff8a5816fd78dce56a
2019-01-15 18:14:27 +00:00
drh
2d77d80a65 Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
2019-01-08 20:02:48 +00:00
dan
c8f44d245c Fix crashes that could occur if SQL NULL values were passed to the built-in
FTS5 snippet function. Edit: breaks amalgamation builds.

FossilOrigin-Name: 6a790b67a0a5c698526db16ea262b13ecdd1b6ca74e80bdccfcad88ddbdc933a
2017-12-26 14:32:25 +00:00
dan
642f0146d7 Fix some compiler warnings in fts5.
FossilOrigin-Name: b066637bb75959267fe6104da323c5112ad78f82
2016-10-05 20:14:29 +00:00
dan
37ff4d4c77 Fix a bug in the fts5 snippet function causing it to return text containing zero phrase instances.
FossilOrigin-Name: b174ed2bde43a1872374ce18b85ac9f067e86fbb
2016-08-24 18:50:47 +00:00
dan
783e778f38 Fix other minor problems with the fts5 snippet() function.
FossilOrigin-Name: b56b2a85cf377686727f3929208ede43f797cf76
2016-08-20 17:23:45 +00:00
dan
7ca1ea16ff Have the fts5 snippet() function avoid favouring snippets that begin with sentences that do not contain search terms. Add an extra bias in favour of the first sentence in the document.
FossilOrigin-Name: d8180af2adfc101dfce95a0498b5cd7b30643b30
2016-08-19 14:25:38 +00:00
dan
e7036202bc Bias the fts5 snippet() function to return snippets that look like they start at the start of sentences.
FossilOrigin-Name: 60de159476edbd48dc363f7f77f09c32ea68422f
2016-08-17 19:05:12 +00:00
dan
8c2b1fde58 Fix a problem in the fts5 snippet() auxiliary function.
FossilOrigin-Name: a861713cc6a3868a1c89240e8340bc7b2b9559da
2016-08-17 11:14:39 +00:00
dan
22e8356368 Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code.
FossilOrigin-Name: bc3f7900d5a06829d123814a5ac7b951bcfc1560
2016-02-11 17:01:32 +00:00
dan
e9eb1593f5 Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode.
FossilOrigin-Name: 72d53699bf0dcdb9d2a22e229989d7435f061399
2016-01-23 18:51:59 +00:00
dan
3e6a141130 Fix some harmless gcc compiler warnings. Mostly in fts5, but also two in the core code.
FossilOrigin-Name: 5d44d4a6cf5c6b983cbd846d9bc34251df8f4bc5
2015-12-23 16:42:27 +00:00
mistachkin
cdabd7bd50 Fix harmless compiler warnings.
FossilOrigin-Name: 1c46c194a2da24fe613d77b5a8d727cc2fc9faa4
2015-10-14 20:34:57 +00:00
dan
351ccb3825 Fix two more warnings in fts5 code.
FossilOrigin-Name: 5343e601fb6f0f24f4e4d5d442f7b5b074734143
2015-10-08 20:28:33 +00:00
dan
ee0c0a8de3 Another change to the fts5 tokenizer API.
FossilOrigin-Name: fc71868496f45f9c7a79ed2bf2d164a7c4718ce1
2015-08-29 15:44:27 +00:00
dan
57e0add3f9 Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document.
FossilOrigin-Name: 90b85b42f2b2dd3e939b129b7df2b822a05e243d
2015-08-28 19:56:47 +00:00
mistachkin
532f179cab Fix some harmless compiler warnings.
FossilOrigin-Name: 3de030c904d125ccf41fa1929646b8a002b5341b
2015-07-14 17:18:05 +00:00
dan
3f09beda45 Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.
FossilOrigin-Name: 7819002ed85497bbd0f9cf4d39df641573324436
2015-07-02 15:52:21 +00:00
dan
7b2ec1ae41 Improve fts5 tests.
FossilOrigin-Name: c1f07a3aa98eac87e2747527d15e5e5562221ceb
2015-04-29 20:54:08 +00:00
dan
6f876c339d Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.
FossilOrigin-Name: 3ee7b5a9f987c269251620ae7cc0fc7876b58ee5
2015-03-07 11:50:31 +00:00
dan
47c467c80e Fix a couple of build problems.
FossilOrigin-Name: a5d5468c0509d129e198bf9432190ee07cedb7af
2015-03-04 08:29:24 +00:00
dan
aacf3d1a3b Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.
FossilOrigin-Name: 65f0262fb82dbfd9f80233ac7c3108e2f2716c0a
2015-01-06 19:08:26 +00:00
dan
ded4f41d1a Tests and fixes for fts5 external content tables.
FossilOrigin-Name: 047aaf830d1e72f0fdad3832a0b617e769d66468
2015-01-05 20:41:39 +00:00
dan
2a615fa627 Fix the fts5 bm25() function so that it matches the documentation.
FossilOrigin-Name: 1ac7a8d0af9a71ddf6a1421033dcb9fa67c6120c
2014-12-23 19:18:34 +00:00
dan
005e10e39a Fixes and simplifications for the snippet() and highlight() functions.
FossilOrigin-Name: ca5d44042aa7461dcc8b700b0763df4df9d4a891
2014-12-22 21:01:52 +00:00
dan
1616d55153 Remove the fts5_test() aux function. Test aux functions using the tcl interface instead.
FossilOrigin-Name: 67e3ffd950c5347d219a06b33ad51949cffa7d90
2014-12-19 20:53:51 +00:00
dan
cb11e73fc2 Add a %_config table to fts5.
FossilOrigin-Name: 83491c56661ca78f96020ba68184bb3fb19e674f
2014-11-27 20:03:45 +00:00
dan
cb62aae034 Add the auxiliary highlight() function to fts5.
FossilOrigin-Name: 059092379f981eb919b500ce447006f9e645fc5a
2014-11-24 16:24:33 +00:00
dan
48d7014067 Fix the customization interfaces so that they match the documentation.
FossilOrigin-Name: fba0b5fc7eead07a4853e78e02d788e7c714f6cd
2014-11-15 20:07:31 +00:00
dan
7b71fbaf93 Add support for savepoints to fts5.
FossilOrigin-Name: 3b19eba042bb2eeb1be60f8d58ebaa0a045d6a5c
2014-08-06 16:30:21 +00:00
dan
937490d4b4 Add further tests for the extension APIs with "ORDER BY rank" queries.
FossilOrigin-Name: 37a417d27e4ebafd4783f62728d7467316b75b17
2014-07-31 11:57:59 +00:00
dan
fdb4a30af2 Add hidden column "rank". Currently this always returns the same value as the bm25() function.
FossilOrigin-Name: 4cc048c3651e830a6aeded924c7f3a60b634e133
2014-07-30 19:41:58 +00:00
dan
92e497e517 Add the "loadfts" program, for performance testing the loading of data into fts3/fts4/fts5 tables.
FossilOrigin-Name: 770b9540c19ad1e3d24adff382332bf032065efd
2014-07-28 20:14:02 +00:00
dan
454b5ce524 Add tests and fixes for bm25() function.
FossilOrigin-Name: 71d32f53e81921e43c933cc968cb1c18d83fe1e0
2014-07-26 18:38:51 +00:00
dan
700b33d7a5 Add extension apis xRowCount, xQueryPhrase, xSetAuxdata and xGetAuxdata. And a ranking function that uses all of the above.
FossilOrigin-Name: c4d50428ab97f77e6721c4f8d03eaaf3ea91f3eb
2014-07-25 20:30:47 +00:00
dan
b30860abe2 Add a snippet() function to fts5.
FossilOrigin-Name: bdc58fd28a63ac9632c3df6c7768a9a236566605
2014-07-23 19:31:56 +00:00
dan
5c1b820460 Add the xTokenize extension API.
FossilOrigin-Name: 8c6b0aff3443fae4b7f0b9adcbf1514992b70653
2014-07-21 14:22:38 +00:00
dan
84d6fb394f Fix the xColumnSize() extension API.
FossilOrigin-Name: 19504c4108472d2ad1281221642b8bd06eb69f4e
2014-07-21 11:44:47 +00:00
dan
e1c77bcfcd Add simple tests for the xColumnText() extension api.
FossilOrigin-Name: 1e9053abdaf5e128d44504ee00dfd909dc25f378
2014-07-19 20:27:54 +00:00
dan
4a165c0af4 Fixes for the xColumnSize() fts5 extension API.
FossilOrigin-Name: 43fcb844726cfeeb1c8a0dbfaa0d2ca22e6ac16c
2014-07-19 15:35:09 +00:00
dan
18689f1bd4 Fixes for tcl list generation in fts5_test().
FossilOrigin-Name: c1f9a4b76c0bbc1ef9f6cdb5d62aa5d536fdf38e
2014-07-16 20:07:59 +00:00
dan
9cfd51f587 Begin adding interface for auxiliary functions.
FossilOrigin-Name: 1e2a7ba0889093416455f488fca893eaeb195d45
2014-07-16 19:15:57 +00:00