Commit Graph

3458 Commits

Author SHA1 Message Date
shess
d53f898f51 Test that terms longer than interior nodes work correctly. A bug
prior to fts2.c r1.10 meant that such large terms caused an eventual
stack overflow. (CVS 3523)

FossilOrigin-Name: 66581162daa188d23078c9d21fd5b2563d4d238a
2006-11-29 21:03:00 +00:00
drh
abcddedb37 Added the speed1.test script (CVS 3522)
FossilOrigin-Name: 30355dfbd920f3b6a78110aaf370371f620324b7
2006-11-29 20:52:59 +00:00
shess
5c327dbb46 http://www.sqlite.org/cvstrac/tktview?tn=2046
The virtual table interface allows for a cursor to field multiple
xFilter() calls.  For instance, if a join is done with a virtual
table, there could be a call for each row which potentially matches.
Unfortunately, fulltextFilter() assumes that it has a fresh cursor,
and overwrites a prepared statement and a malloc'ed pointer, resulting
in unfinalized statements and a memory leak.

This change hacks the code to manually clean up offending items in
fulltextFilter(), emphasis on "hacks", since it's a fragile fix
insofar as future additions to fulltext_cursor could continue to have
the problem. (CVS 3521)

FossilOrigin-Name: 18142fdb6d1f5bfdbb1155274502b9a602885fcb
2006-11-29 05:17:28 +00:00
shess
7e3d0c2d2f Delta-encode terms in interior nodes. While experiments have shown
that this is of marginal utility when encoding terms resulting from
regular English text, it turns out to be very useful when encoding
inputs with very large terms. (CVS 3520)

FossilOrigin-Name: c8151a998ec2423b417566823dc9957c7d5d782c
2006-11-29 01:02:03 +00:00
drh
3590f15775 Improvements to the speed tests recently added to the test suite. (CVS 3519)
FossilOrigin-Name: 272c1a6e61d053121b5412564948dad4366b5727
2006-11-23 21:09:10 +00:00
drh
2fca7fef28 Fix a typo in the OMIT_AUTORIZATION in expr.c. (CVS 3518)
FossilOrigin-Name: 3dea7fbefdadb269e58ae76bb9a7281c96d8b15b
2006-11-23 11:59:13 +00:00
drh
600123299e Make the pager sector size configurable at compile-time. (CVS 3517)
FossilOrigin-Name: 6221cf4f52e79f53043b7c10daedf2755d8fb167
2006-11-23 11:58:44 +00:00
drh
b62c335ed6 Add performance tests to the test suite. (CVS 3516)
FossilOrigin-Name: 270c745dffad7aa24f8707720b3d68ad7a6b2749
2006-11-23 09:39:16 +00:00
drh
45e29d8a1e Fix the ".dump" command in the shell. Ticket #2072. Also ticket #2065. (CVS 3515)
FossilOrigin-Name: 9fdc249609a4745715a2bf49bbf1376ea243a20a
2006-11-20 16:21:10 +00:00
drh
c28e3a1828 Make sure VACUUM cleans up after itself. Ticket #2071. (CVS 3514)
FossilOrigin-Name: 2fdc147d0059dcdfff2da33bd9fedb0bee057aa1
2006-11-18 20:20:21 +00:00
shess
f72442be68 Store minimal terms in interior nodes. Whenever there's a break
between leaf nodes, instead of storing the entire leftmost term of the
rightmost child, store only that portion of the leftmost term
necessary to distinguish it from the rightmost term of the leftmost
child. (CVS 3513)

FossilOrigin-Name: f6e0b080dcfaf554b2c05df5e7d4db69d012fba3
2006-11-18 00:12:44 +00:00
shess
9e6a561554 Refactoring groundwork for coming work on interior nodes. Change
LeafWriter to use empty data buffer (instead of empty term) to detect
an empty block.  Code to validate interior nodes.  Moderate revisions
to leaf-node and doclist validation.  Recast leafWriterStep() in terms
of LeafWriterStepMerge(). (CVS 3512)

FossilOrigin-Name: f30771d5c7ef2b502af95d81a18796b75271ada4
2006-11-17 21:12:15 +00:00
shess
de163af26e Delta-encode docids. This is good for around 22% reduction in index
size with DL_POSITIONS.  It improves performance about 5%-6%. (CVS 3511)

FossilOrigin-Name: 9b6d413d751d962b67cb4e3a208efe61581cb822
2006-11-13 21:09:24 +00:00
shess
debbcdfead Require a minimum fanout for interior nodes. This prevents cases
where excessively large terms keep the tree from finding a single
root.  A downside is that this could result in large interior nodes in
the presence of large terms, which may be prone to fragmentation,
though if the nodes were smaller that would translate into more levels
in the tree, which would also have that problem. (CVS 3510)

FossilOrigin-Name: 64b7e3406134ac4891113b9bb432ad97504268bb
2006-11-13 21:00:54 +00:00
shess
545311eeca Allow backing tables to be missing on dropping fts table. Fixes
http://www.sqlite.org/cvstrac/tktview?tn=1992,35 . (CVS 3509)

FossilOrigin-Name: 9628a61a6f33b7bec3455086534b76437d2622b4
2006-11-13 20:15:27 +00:00
aswift
448aa6f71a The uninitialized file descriptor from the unixFile structure is passed to sqlite3DetectLockingStyle in allocateUnixFile rather than the file descriptor passed in. This was causing the locking detection on NFS file systems to behave somewhat randomly and the result was locks were not respected and data loss could occur. (CVS 3508)
FossilOrigin-Name: b9dd16ef3df0785d8d0eea9a6f2ad580a7289834
2006-11-11 01:31:58 +00:00
drh
1fe336f69c Update the documentation to talk about the new sqlite3_prepare_v2() API. (CVS 3507)
FossilOrigin-Name: d9e14b6121a7b6786cccafd6e1e83474554632aa
2006-11-09 15:18:00 +00:00
drh
b900aaf34a First cut at adding the sqlite3_prepare_v2() API. Test cases added, but
more testing would be useful.  Still need to update the documentation. (CVS 3506)

FossilOrigin-Name: f1efae9224170c9155afcf17ab3ee769a557b874
2006-11-09 00:24:53 +00:00
drh
47ad6840f5 Make the .exit and .quit commands work again in the shell. Ticket #2056. (CVS 3505)
FossilOrigin-Name: f39978ef13e986a16ee322ee84ab9bd38ffc5a8b
2006-11-08 12:25:42 +00:00
drh
f4fa85826f Fix a typo in os_win.c. Ticket #2055. (CVS 3504)
FossilOrigin-Name: d309680ec7a806d181b601e0105aebf1e33bfb81
2006-11-07 15:02:07 +00:00
drh
551b77365e Use the difference between the SQLITE_IOERR_SHORT_READ and SQLITE_IOERR_READ
returns from sqlite3OsRead() to make decisions about what to do with the
error. (CVS 3503)

FossilOrigin-Name: 6324ea811eec1200cee89e6f377368eaf2fcda77
2006-11-06 21:20:25 +00:00
drh
382765866b Fix a bug in the optimizer that was causing it to miss an OR optimization
opportunity. (CVS 3502)

FossilOrigin-Name: 9bf153b54c6e9ba16914dedd9e2949f32d7550ea
2006-11-06 15:10:05 +00:00
drh
0a9e376c55 Modify Makefile.in to install sqlite3ext.h. (CVS 3501)
FossilOrigin-Name: 35c8c4781736d45019d8b823b8517c24622d3313
2006-11-01 12:20:16 +00:00
drh
e804cebacc Remove dead code from btree.c. Ticket #2050. (CVS 3500)
FossilOrigin-Name: dbd0125c62457681689db48e1f0a752767855773
2006-11-01 12:08:40 +00:00
drh
fd288f3549 Change the default temp file prefix to be "sqlite" spelled backwards.
Tickets #2049 et al. (CVS 3499)

FossilOrigin-Name: a19ce5c1c4e86d141f7e04af6babeb2ecb75b523
2006-10-31 21:27:33 +00:00
drh
79de864cd1 Change the default prefix for temporary files so that it no longer
contains the text "sqlite".  In this way, perhaps we will not get so
many false bug reports such as ticket #2049, #1989, and #1841. (CVS 3498)

FossilOrigin-Name: 7ce48000bb0dafda8a171bfc040dfe2300f84ed2
2006-10-31 21:16:48 +00:00
shess
aedbce0376 Fix a pair of memory leaks. These were turned up by running valgrind
memcheck with various 10k doc insert, update, delete, and query tests. (CVS 3497)

FossilOrigin-Name: 3cd9b64b96018f69163ad0be0b5c07dd1be6abc6
2006-10-31 18:13:42 +00:00
drh
2e7568f237 Make the command-line shell ".dump" command more resilient in the face of
database corruption. (CVS 3496)

FossilOrigin-Name: ebd44f0b5e229b990910bb44565de9a5c06cb5bc
2006-10-31 18:08:27 +00:00
drh
371de5ad8b Changes to support non-ASCII characters in win95 filenames.
Ticket #2047. (CVS 3495)

FossilOrigin-Name: 9fa3ae584ae4936696fd2f23a64697f0409ef313
2006-10-30 13:37:22 +00:00
drh
50b3996899 Enhance the optimizer so that IS NULL can use an available index. (CVS 3494)
FossilOrigin-Name: 64762a9d582e4655d6bc5989d8e0ad773d659a7d
2006-10-28 00:28:09 +00:00
drh
89b5cad3c2 Fix the ".dump" command in the command-line shell so that it shows
TRIGGERs and VIEWs.  Ticket #2044. (CVS 3493)

FossilOrigin-Name: 58171a41f706dd2fab1da5d83d2176d0103643fb
2006-10-27 14:21:54 +00:00
drh
0660e26efe Changes directed toward optimizing IS NULL terms in WHERE clauses. (CVS 3492)
FossilOrigin-Name: 4d336e9ef5f65b95959e7d01cd0357d46e9b1fa5
2006-10-27 14:06:57 +00:00
drh
c49f44eff1 Bring CVS output into more commonly accepted practice. Tickets #2030, #1573.
Add command-line options -bail and ".bail" commands.  Default behavior is
to continue after encountering an error.  Ticket #2045. (CVS 3491)

FossilOrigin-Name: 517712d6fbc5ba5299942a54852298030f4d3381
2006-10-26 18:15:42 +00:00
drh
c28490c056 Command-line shell enhancements. Bail out when errors are seen in
non-interactive mode.  Override isatty() using -interactive or -batch
command-line options.  Report line number in error messages.
Tickets #2009, #2045. (CVS 3490)

FossilOrigin-Name: 3baa04cfb91039e27f642f6f78ef761b5770cb08
2006-10-26 14:25:58 +00:00
shess
93d2a81401 Empty queries should get no results. My recent change
( http://www.sqlite.org/cvstrac/chngview?cn=3486 ) broke test fts2a-5.3.
This change should make the expected result more obvious. (CVS 3489)

FossilOrigin-Name: cde383eb467de0d752e94a22cd2f890c2dc599cc
2006-10-26 00:41:51 +00:00
shess
9d5586fc9f Make memset() uses less error-prone.
http://www.sqlite.org/cvstrac/tktview?tn=2036,35 describes some cases
where we were passing memset() a length which was the sizeof a
pointer, rather than the structure pointed to.  Instead, wrap this
idiom up in CLEAR() and SCRAMBLE() macros. (CVS 3488)

FossilOrigin-Name: 5878add0839f9c5bec77caae2361ec20cb60b48b
2006-10-26 00:04:31 +00:00
shess
627a74c48c Remove unreferenced local variable. (CVS 3487)
FossilOrigin-Name: 2d3b22197c7c06488b789cce333b34b6d1ae39aa
2006-10-25 23:22:03 +00:00
shess
87f1d16bdb Replace the DocList and DocListReader structures. The new structures
distinguish reading from a static buffer from writing to a dynamic
buffer.  This allows n-way doclist merging, and in-place merging of
segment leaf nodes, which together cut segment merge times in half. (CVS 3486)

FossilOrigin-Name: af5bfb986e39248abbfc6fff2e13c6f9e634a751
2006-10-25 21:00:09 +00:00
shess
d5d63ca6a3 Test to force edge cases in query logic. Basically, exercise code to
handle lack of hits correctly. (CVS 3485)

FossilOrigin-Name: 2cb59033662f25677169e2e63b871fb0a4c10c21
2006-10-25 20:27:39 +00:00
shess
9289cba076 Don't store empty segments. When inserting empty strings, the code
was writing out a segment made up of a single leaf node containing the
\0 header.  LeafReader assumed that leaf nodes always contained at
least one term, so assertions would fail.

While it would be possible to support reading and merging empty
segments, there's no reason to do so.  While this change could have
been done in writeZeroSegment(), I put it in leafWriterFlush() so that
it would work right if segmentMerge() created an empty segment, which
could happen with future changes to how deleted documents are handled. (CVS 3484)

FossilOrigin-Name: fed79beec7da24a26ae94494bdc0c98dd102bc06
2006-10-25 05:21:55 +00:00
shess
8e734e3518 dup fts1 tests and edit for fts2. fts1porter.test omitted because it
depends on being able to poke through to %_term table. (CVS 3482)

FossilOrigin-Name: 2806c3415841b98d52163fa672a747d0a47ffd41
2006-10-19 23:36:25 +00:00
shess
1c5f160762 Add tests for delete and update in fts1. (CVS 3481)
FossilOrigin-Name: b01c4371d861a087623a34c38cb8db74eca95348
2006-10-19 23:28:35 +00:00
shess
a26cf57782 fts2 support for testing. These are a prelude to adding some test
scripts. (CVS 3480)

FossilOrigin-Name: 004ad1943f8c5933ee9584a57b2de9d421470d3c
2006-10-19 20:27:58 +00:00
drh
b3fa0e0146 When converting UTF8 or UTF16 strings, change overlong strings and other
illegal codes to 0xFFFD.  Ticket #2029. (CVS 3479)

FossilOrigin-Name: 0c6736df9cb4c3c8f6224e30df939cead9cd5369
2006-10-19 01:58:43 +00:00
drh
28f4591466 Fix a problems that arise if malloc() fails while compiling SELECT
statements within a TRIGGER. (CVS 3478)

FossilOrigin-Name: ee4894b49995e4904db1991281563cfbb7b1c16d
2006-10-18 23:26:38 +00:00
drh
f6bbe022c7 Make sure the names of all expressions in compound SELECT statements used
as subqueries are correctly resolved.  Ticket #2018. (CVS 3477)

FossilOrigin-Name: b886eaa334150262ce4d1a1d0470ca4cf623a396
2006-10-13 15:34:16 +00:00
drh
5048962a0f Avoid expanding %d contained in the action of a lemon parser rule.
Ticket #1063.  This is a fix for lemon only.  It does not effect SQLite. (CVS 3476)

FossilOrigin-Name: 81daedcf48372949b9df009ce6121a514ecf6f2e
2006-10-13 12:25:29 +00:00
drh
d9033a6569 Removing debugging printf from the porter stemmer code. Ticket #2016. (CVS 3475)
FossilOrigin-Name: 7a08c6272f76d53b13313019b4f9da3c8f02b650
2006-10-13 11:55:39 +00:00
shess
8a235d4d3b Convert fts2 to store data in a way which allows for much faster
updates.  Groups of documents form segments which are encoded in a
btree layered over a table of blocks, with various tricks to make
merges fast.  This performs 20x-25x faster than fts1 when loading the
Enron corpus, and is only slightly slower for queries. (CVS 3474)

FossilOrigin-Name: 85272b2f5394e37916afb1d509e7296810d976f5
2006-10-12 23:15:24 +00:00
aswift
108bc320f9 Bug fix: named local variable lockStyle as lockingStyle in SQLITE_ENABLE_LOCKING_STYLE block in allocateUnixFile (CVS 3473)
FossilOrigin-Name: aa0b96c3dfa1d8008cd944281c227604b2ee16d1
2006-10-11 17:19:46 +00:00