Commit Graph

512 Commits

Author SHA1 Message Date
mistachkin
c197eedbb4 Fix several compiler warnings seen with MSVC.
FossilOrigin-Name: 8b5f617c2a5f0bf7bdbd9144aee58d69564db1e2
2012-10-01 20:12:30 +00:00
mistachkin
d5578433ff Fix all known instances of 'repeated the' style typos in comments. No changes to code.
FossilOrigin-Name: 9b19b847533f944f289d93dcdba29c0d67bf251c
2012-08-25 10:01:29 +00:00
dan
4dc3d73d69 Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option.
FossilOrigin-Name: e799222f3b8246e65657a758437914ece7069ba9
2012-08-20 17:24:48 +00:00
mistachkin
fe7b2bc9ff Merge several compiler compatibility fixes to trunk.
FossilOrigin-Name: d3d491a5461e21d84c6425977b80deedc7fa8a3a
2012-06-21 14:05:15 +00:00
dan
2ae26b759d Fix a problem with identifying white-space characters outside of the ascii range in the ICU tokenizer.
FossilOrigin-Name: 892b74116a3b23268895b96433d18ef00c1433d8
2012-06-18 20:52:32 +00:00
mistachkin
c8bde37d19 Fix a few compilation issues that can occur with certain compilers (e.g. GCC 2.95.3, MSVC).
FossilOrigin-Name: f970a3de61fe2ebaf1778c4a3383cfdc52299162
2012-06-18 08:00:56 +00:00
drh
1c30debd41 Fix an obscure memory leak in FTS3 that can come about when two memory
allocations are immediately adjacent to one another.

FossilOrigin-Name: 025227be5495f950c466dfabac140cba69e498be
2012-06-08 14:01:53 +00:00
dan
25cdf46ae4 Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer.
FossilOrigin-Name: e56fb462aa1f11bb23303ae0dc62815c21e26a52
2012-06-07 15:53:48 +00:00
dan
2c897e3e5f Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61.
FossilOrigin-Name: eccd6b6580637084495b80e9232262188ba0cf8d
2012-06-06 19:51:27 +00:00
dan
754d3adf7c Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0".
FossilOrigin-Name: 790f76a5898dad1a955d40edddf11f7b0fec0ccd
2012-06-06 19:30:38 +00:00
drh
0e3037ac36 Refactor the float-to-double rounding routines so that they compile without
warnings.

FossilOrigin-Name: f607ad27c1ede27af24dbee10ca867c8f7761ee3
2012-05-29 00:30:43 +00:00
drh
c6bff38216 Simplification to the coordinate rounding logic in RTree.
FossilOrigin-Name: df24072de27555c6b530b02e821ea8b066c554fc
2012-05-28 20:16:42 +00:00
drh
7923863602 When converting 64-bit floating point coordinates to 32-bit in RTree, take
care to round the values such that the size of the bounding box is enlarged.

FossilOrigin-Name: f4e8ff03eae70334632455a867859cfcc25682be
2012-05-28 19:19:25 +00:00
drh
a9cfaba95a Omit the fts3 unicode character class routines from the build if fts3/4
is disabled.

FossilOrigin-Name: c00bb5d4601efc15933f222349e96a043b610a19
2012-05-28 12:22:00 +00:00
dan
7946c53009 If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer.
FossilOrigin-Name: e71495a817b479bc23c5403d99255e3f098eb054
2012-05-26 18:28:14 +00:00
dan
501c74d3e1 Change the format of the tables used by sqlite3FtsUnicodeTolower() to make them a little smaller.
FossilOrigin-Name: b89d3834f6690073fca0fc22c18afa1fb280ea7d
2012-05-26 17:57:02 +00:00
dan
cf9f6f1375 Add fault-injection tests that use the unicode61 tokenizer. Fix a problem revealed by the same.
FossilOrigin-Name: ed28c48a3dd7e766e60db0d96ef5460bf9913e6b
2012-05-26 15:44:08 +00:00
dan
ab322bd21e Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1.
FossilOrigin-Name: 8f3e60aa2253f21bcee5d03982cfdd7f16c00060
2012-05-26 14:54:50 +00:00
dan
1c7016c9a5 Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range.
FossilOrigin-Name: cf7b25d47687635a04f4347d45f135c686b9d758
2012-05-25 19:50:12 +00:00
dan
80ed5a56a5 Fix comments in generated file fts3_unicode2.c.
FossilOrigin-Name: 3dc567ef4702d9a63d78d11ff705cb7f7359f7a6
2012-05-25 18:48:48 +00:00
dan
3d403c71a8 Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators.
FossilOrigin-Name: 0c13570ec78c6887103dc99b81b470829fa28385
2012-05-25 17:50:19 +00:00
dan
5cfed52dd1 Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38].
FossilOrigin-Name: 4d3e1673b309a3d6a6cac01d67dfd79a8007f991
2012-05-10 17:43:14 +00:00
drh
83cc139231 Fix harmless compiler warnings on x64 MSVC, mostly in test code, but also in
tclsqlite.c and in the FTS4 module.

FossilOrigin-Name: 3281972eaa46cb57fd9f0387063f47430dc0a3b4
2012-04-19 18:04:28 +00:00
drh
85e7243acf Fix harmless static-analysis warnings, mosting having to do with memory
leaks in the command-line shell.  Add a clang analysis of the command-line
shell to the "warnings-clang.sh" script.  Other minor cleanups to the
command-line shell code.

FossilOrigin-Name: 93a0f452a7023898ad3d62ee81b39a80477c332f
2012-04-11 11:38:53 +00:00
drh
2aad3da6ea Fix a typo in the rtree6.test script that prevented it from running.
FossilOrigin-Name: 221fe4a8ea5bea90031e459746ea71ff173e6f52
2012-04-03 17:05:16 +00:00
drh
f439fbdab5 The SQLITE_RTREE_INT_ONLY compile-time option causes the RTree extension
to use only integer math and store only integer coordinates.

FossilOrigin-Name: 02b7640f5118e0a635b68f65765191bb3171b7bd
2012-04-02 21:35:42 +00:00
drh
3b06a2a056 Add #ifdefs to allow a test build to succeed even if SQLITE_ENABLE_FTS3 is
not defined.

FossilOrigin-Name: fb121980e48af368353431fd04924e414b65c852
2012-04-02 17:18:23 +00:00
dan
a25a42cfe7 Change the ICU tokenizer so that it does not attempt to call strlen(NULL).
FossilOrigin-Name: 04298f1ac42c40cb2a48092b415acf96a08954b7
2012-03-31 11:58:23 +00:00
drh
7da5fcb0b7 Fix MSVC compiler warnings in test code.
FossilOrigin-Name: cb7a850439c9a4a7887650d6b81d95ab8025de5b
2012-03-30 14:59:43 +00:00
drh
4e245a4c35 Fix compiler warnings on GCC and MSVC and fix a C89-ism that
broke the build for MSVC.

FossilOrigin-Name: b451c0f97f0abe78ebe6c62ff489ec1ad8a1f767
2012-03-30 00:00:36 +00:00
drh
cdb86dc437 Add output of PRAGMAs auto_vacuum and encoding to the "schema" command of the
fts3view utility program.

FossilOrigin-Name: e31076319363a46905836880765bae3bf204ed19
2012-03-27 18:00:05 +00:00
dan
311ec02587 Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables.
FossilOrigin-Name: 5c447e226afca0d46b9ed994dea26a16a9ae168c
2012-03-27 15:00:06 +00:00
drh
a1a9f0a0d8 In the fts3view utility, label the blank segments used to mark the end of a
segment sequence for a level/idx as "null".  Improve the alignment of root
segment names.

FossilOrigin-Name: 04aea0245e4183fef3664609f5a6353b65d71a85
2012-03-27 14:54:44 +00:00
drh
1ecb59a1e5 Enhance the fts3view tool with the big-segment command and fix a bug in the
display of doclists.

FossilOrigin-Name: e9436d8038e5a0d1ba992a77d1064d4a55595f57
2012-03-27 13:51:31 +00:00
dan
5da0aa1603 Allow multiple incremental merges to proceed concurrently. This is required to prevent a large crisis-merge from occuring while an even larger incremental-merge is underway.
FossilOrigin-Name: 7ed9d2f24a650b424b97dfc19b8042c4cf09c82c
2012-03-27 11:48:02 +00:00
drh
40aab4c56d Minor correct errors in the file format description for FTS3/4 contained in
the fts3.c header comment.

FossilOrigin-Name: fb8aacdd8fbdc946cb271cc589f76b806387937d
2012-03-27 00:38:33 +00:00
drh
280d5c33cd Enhance fts3view to show decodes of segments and doclists.
FossilOrigin-Name: 6d09de231b68dd9520d99c65d133f26e90eb784f
2012-03-27 00:34:04 +00:00
drh
36f6b891e6 Add the fts3view utility program.
FossilOrigin-Name: f936c8ea16d21345fd1622272dc7e9850acb2493
2012-03-26 21:57:53 +00:00
drh
cbf9450404 Fix FTS3 so that it works even without SQLITE_DEBUG.
FossilOrigin-Name: a18c103121529c2e3c6a8ada16a4c40d14080670
2012-03-26 14:36:42 +00:00
dan
6f4df8ac83 Modify the FTS integrity-check so that the checksums do not depend on the results of signed integer overflow, which is undefined in C.
FossilOrigin-Name: f907fc3fb387e74bb66babcbf050748cb253a6fa
2012-03-26 10:57:31 +00:00
dan
84bce14f77 Add a comment to explain how the FTS integrity-check works.
FossilOrigin-Name: 64e8a116f39434a3b7347f01a47f88eef3276742
2012-03-26 10:47:03 +00:00
dan
cbcd9f5357 Add an experimental integrity-check function to FTS.
FossilOrigin-Name: 40fc8804743dfb005991e9c5ef7b0ebcb3c2e731
2012-03-26 10:36:55 +00:00
drh
6c2e7e19fa Enable fts3 tables to use incremental merge by automatically creating the
%_stat table when it is needed.

FossilOrigin-Name: cc051fc0b2d89603b27b94cf2afdbda417ee9d94
2012-03-24 17:29:05 +00:00
dan
61fa09dbf2 Fix a failing assert() in the FTS3_LOG_MERGES related code.
FossilOrigin-Name: 4220d52cb3426f1680b72d57ecc9f4ade029357d
2012-03-24 17:09:11 +00:00
dan
0dfcdeb4f0 Add a comment to fts3SyncMethod() to justify the nMinMerge=64 constant.
FossilOrigin-Name: af55ca5fc6778cb6d1a79a17dfa2d4e567ea1ccc
2012-03-24 16:43:55 +00:00
drh
790964ebc4 Fix a bug in debugging code enabled when FTS3_LOG_MERGES is defined.
FossilOrigin-Name: 2e06babf49de844d0e878d68114dbcc5ad4d6e54
2012-03-24 16:18:08 +00:00
dan
45eddd6855 Remove the Fts3Table.mxLevel variable.
FossilOrigin-Name: 67a0cffc9d07be7f09dad9d019a18160711295cd
2012-03-24 16:11:21 +00:00
dan
4b1e4dabc3 Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code.
FossilOrigin-Name: 0d841c957c6ec4afecb49504177c6279e09d7012
2012-03-24 14:45:19 +00:00
drh
4ef9dfff97 An attempt at automatic incremental merging for FTS4.
FossilOrigin-Name: ed69434cd89084f4b57bd2cc4f5cc558904af565
2012-03-24 02:20:43 +00:00
dan
d59de73e27 Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation.
FossilOrigin-Name: ed7c17ea165f6348506bd23ebc58c427bb65d697
2012-03-23 18:26:11 +00:00