dan
|
7946c53009
|
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
|
2012-05-26 18:28:14 +00:00 |
|
dan
|
7a796731db
|
Add coverage tests for fts3_unicode.c.
FossilOrigin-Name: 07d3ea8a3cb179fab6c48934fc6751f53b507d36
|
2012-05-26 16:22:56 +00:00 |
|
dan
|
ab322bd21e
|
Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.
FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060
|
2012-05-26 14:54:50 +00:00 |
|
dan
|
3d403c71a8
|
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
|
2012-05-25 17:50:19 +00:00 |
|