dan
|
3fb7bd5ee0
|
Fix sanitizer complaint in fts3 code.
FossilOrigin-Name: 424b39eaeb578f85e58214af4b467e6e363431308ab7a327774e62fa70d3c2db
|
2021-03-10 20:04:21 +00:00 |
|
drh
|
2d77d80a65
|
Use 64-bit math to compute the sizes of memory allocations in extensions.
FossilOrigin-Name: ca67f2ec0e294384c397db438605df1b47aae5f348a8de94f97286997625d169
|
2019-01-08 20:02:48 +00:00 |
|
dan
|
e89feee5c3
|
Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5
and FTS3/4.
FossilOrigin-Name: 06177f3f114b5d804b84c27ac843740282e2176fdf0f7a999feda0e1b624adec
|
2018-12-03 16:14:49 +00:00 |
|
dan
|
920c83f18f
|
Fix some problems in fts3 found by address-sanitizer.
FossilOrigin-Name: 16a8e84fa7f67a467f824bdd7f72cbd6a6e95dab8cc7aa1e0e751720b98f3e31
|
2017-03-20 18:53:32 +00:00 |
|
drh
|
490fe86f1a
|
Fix harmless compiler warnings.
FossilOrigin-Name: 52b03f045edf6fc29f9ba9a5cac53a59f0feb0eb
|
2014-08-11 14:21:32 +00:00 |
|
mistachkin
|
86ac612e8a
|
Fix some harmess compiler warnings in the FTS3 Unicode module.
FossilOrigin-Name: c01caea5d6ad7a570628b85fb3056ac955ff03dd
|
2014-07-19 15:44:29 +00:00 |
|
dan
|
2eaf03d72b
|
Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE.
FossilOrigin-Name: 0cc0230ae9cfc9760fd8ef2c25e82576b052dbeb
|
2014-07-03 12:18:22 +00:00 |
|
mistachkin
|
48864df97d
|
Many spelling fixes in comments. No changes to code.
FossilOrigin-Name: 6f6e2d50941e444ebc83604daddcc034137a05b7
|
2013-03-21 21:20:32 +00:00 |
|
dan
|
25cdf46ae4
|
Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.
FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc62815c21e26a52
|
2012-06-07 15:53:48 +00:00 |
|
dan
|
2c897e3e5f
|
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
|
2012-06-06 19:51:27 +00:00 |
|
dan
|
754d3adf7c
|
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
|
2012-06-06 19:30:38 +00:00 |
|
dan
|
7946c53009
|
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
|
2012-05-26 18:28:14 +00:00 |
|
dan
|
1c7016c9a5
|
Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range.
FossilOrigin-Name: cf7b25d47687635a04f4347d45f135c686b9d758
|
2012-05-25 19:50:12 +00:00 |
|
dan
|
3d403c71a8
|
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
|
2012-05-25 17:50:19 +00:00 |
|