Commit Graph

12170 Commits

Author SHA1 Message Date
drh
6b5d4759ce Change vdbeSorterExtendFile() so that it makes a best effort to create the
PMA file of the desired size, but does not return an error if unable.

FossilOrigin-Name: 217814bc4b53fab7bdad433e24e8aef8998c38fe
2014-04-04 02:13:26 +00:00
drh
1aca5caa88 Merge all recent changes from trunk.
FossilOrigin-Name: a0910079adde95245680dee59b43613b60903f10
2014-04-03 16:42:21 +00:00
drh
c438df1be0 Use OP_Copy instead of OP_SCopy when moving results out of a subquery,
to prevent the subquery results from changing out from under the outer
query.  Fix for ticket [1e64dd782a126f48d78].

FossilOrigin-Name: d5513dfa23baa0b0a095aaf17d19aacd30dcef61
2014-04-03 16:29:31 +00:00
dan
8930c2ab0c Fix an integer overflow problem in the sorter.
FossilOrigin-Name: 9d3351b8d713232133dad149c73fb2a27c72abb1
2014-04-03 16:25:29 +00:00
drh
831116d164 Fix a typo in the "Synopsis:" comment for the OP_VFilter opcode.
FossilOrigin-Name: 48ecdd4aff03741f96c070dced69c3c273b652cb
2014-04-03 14:31:00 +00:00
dan
6e4cc55e1f Fix minor errors causing compilation to fail with SQLITE_MAX_WORKER_THREADS set to a value greater than zero.
FossilOrigin-Name: 0561272abf357a2f4709f6c02866e570d19cd344
2014-04-03 14:29:08 +00:00
drh
a634fb1577 Refactor local object and method names in vdbesort.c so that their names
more closely reflect their actual use.

FossilOrigin-Name: d284e30eb1db144965fa85566e4234e30464350b
2014-04-03 02:54:27 +00:00
drh
54d0d2dd9a In the command-line shell, run set writable_schema before running the
".clone" command.

FossilOrigin-Name: 9d2ae6342c8afa904bec591ebe134ff7f536b71c
2014-04-03 00:32:13 +00:00
drh
ac4f0039c0 Add a big introductory comment to vdbesort.c explaining its operation at a
high level.  Also adjust some symbolic names and fix other comment issues in
that file.

FossilOrigin-Name: eef60f1bf54fcdc7b32f96ebb87a9a0bf0776e8b
2014-04-02 18:58:49 +00:00
dan
dd95d30f82 Fix some problems with OOM handling in vdbesort.c.
FossilOrigin-Name: 47e702bd8392bc50c4edaf6a2c8c499af87b520e
2014-04-02 15:15:25 +00:00
drh
8d8f56296b Change the name of the SorterThread object to "SortSubtask" to avoid confusion
with the SQLiteThread object.

FossilOrigin-Name: 4ee2d910fbbed8d4def15e4e99ee225839f3a739
2014-04-02 14:38:14 +00:00
dan
fad9f9a8a6 When sorting data for a CREATE INDEX statement in single-threaded mode, assume that keys are delivered to the sorter in primary key order. Also fix various comments that had fallen out of date.
FossilOrigin-Name: 821d1ac4504243fa13b9e3c0d56361ad9fb80d78
2014-04-01 18:41:51 +00:00
dan
578e1ca8d7 Even if compile time option SQLITE_MAX_WORKER_THREADS is set to one or greater, set the default number of worker threads to zero. Distribute data more evenly between threads in sqlite3VdbeSorterWrite() to improve performance when sorting large amounts of data. Add new test file sort2.test.
FossilOrigin-Name: 643c86a056168e39fcb7f39b8a72731f1eb246db
2014-04-01 15:38:44 +00:00
dan
38fdead890 Fix a problem with OOM handling in the sorter code.
FossilOrigin-Name: 59cd5229e2b5be5272cf57c7e7d09e97d16a5425
2014-04-01 10:19:02 +00:00
dan
b3f56fdb69 Add the SQLITE_MAX_WORKER_THREADS compile time option. And the SQLITE_CONFIG_WORKER_THREADS sqlite3_config() switch.
FossilOrigin-Name: 2774710df8cd2bfaca49888c69f1b01c0ddadf9a
2014-03-31 19:57:34 +00:00
drh
b663587890 Avoid a (harmless) buffer overread that is possible on an OOM when
MEMSYS5 is engaged.

FossilOrigin-Name: b3296267fb67b9f59719a37093253062edde3746
2014-03-31 13:42:42 +00:00
dan
853c4a7621 Fix a broken assert() in vdbesort.c.
FossilOrigin-Name: 18d1b402f2dbe78f1a1113bb356b710e348365ef
2014-03-29 10:01:58 +00:00
dan
e7c84cc7e3 Fix a problem in vdbesort.c causing spurious SQLITE_NOMEM errors when using memsys3 or memsys5.
FossilOrigin-Name: a683c05f6250389e84b980b16559e162ba1a27c2
2014-03-29 09:34:45 +00:00
dan
ff9fce4d60 Add the optimization to avoid some unnecessary calls to sqlite3VdbeRecordUnpack() added to the trunk by [707ea170b3].
FossilOrigin-Name: fc4d04e6b039ea5aeb47739e38c5926e63a4b01b
2014-03-29 06:27:35 +00:00
dan
2f17001521 Merge latest changes from orderby-planning branch.
FossilOrigin-Name: 4c7fb5423430f3b936befaa7c309f8e1968ee7d8
2014-03-28 19:18:16 +00:00
drh
7c8e9c78c8 Merge the latest changes from trunk.
FossilOrigin-Name: 3047a25f1c41e83f0b4772f7c36fbfec0f12dc7e
2014-03-28 18:35:39 +00:00
drh
70c93204ea Disable the wal64k.test script for non-unix systems since it depends on
unix-only features.

FossilOrigin-Name: 27deb6e49bcc76714dbdc61b34748603155ac770
2014-03-28 14:41:35 +00:00
drh
36ed03428d Fix a harmless compiler warning.
FossilOrigin-Name: a4e47150f32b3a4120b1f89ccc66d633d829e3bb
2014-03-28 12:56:57 +00:00
drh
a1f7c0a21c Enhance the sqlite3VdbeRecordCompare() routines so that if they encounter
database corruption, they will set the UnpackedRecord.isCorrupt field and
return 0.  The sqlite3BtreeMovetoUnpacked() routine detects this and returns
SQLITE_CORRUPT, causing the corruption to be reported back to the top-level.

FossilOrigin-Name: 7fa85eaaaf6d211378620d728a759fdfe30a15b0
2014-03-28 03:12:48 +00:00
dan
6971952c65 Instead of allocating a single large buffer at the beginning of each sort operation, start with a small buffer and extend it using realloc() as required.
FossilOrigin-Name: 81987c8ceb64f051528a6ca42673821d9ab7c0ff
2014-03-27 19:25:02 +00:00
drh
27de5c5cb6 Minor cleanup of the code in the query planner that computes the costs
estimates for the various plans.  There are no changes to the costs at this
time.  But the code is slightly more readable now and that might facilitate
future enhancements.

FossilOrigin-Name: 9b4d7226bcee38be5ac68a54bee03b4179cb69fc
2014-03-27 18:36:34 +00:00
dan
face087212 Use xFetch() to access temporary files in vdbesort.c. Use a single large allocation instead of many small allocations when accumulating records in vdbesort.c. This is an interim commit - it allocates a buffer the size of the page-cache every time data is sorted.
FossilOrigin-Name: f4ac1bf28c4ba395ccab8f1c9df72614a61095a7
2014-03-27 17:23:41 +00:00
drh
382bdeabef Enhance the logest.c utility with new operators: "dup", "inv", "log", and
"nlogn".  Provide help on an invalid input.

FossilOrigin-Name: b4bd2a062c4baf5f622d61b7411f00de5904ef56
2014-03-27 14:05:38 +00:00
drh
bfa463696b Merge the latest trunk changes and the fix for the crash on a corrupt
database.

FossilOrigin-Name: 0b35346c32dba14963c85ec178f2b46aa2bbf6dc
2014-03-27 00:09:00 +00:00
drh
664eb14d8e Merge from trunk the fix for the crash on a corrupt database.
FossilOrigin-Name: 8cb2b02baa7ef9aa96319e977f0315328f944237
2014-03-26 19:45:01 +00:00
dan
68916c9fd3 Add an extra test case for the potential buffer overread patched by [28ddecff04].
FossilOrigin-Name: f585f5d7a0f9bf8c590388654a3638231eba8892
2014-03-26 15:14:59 +00:00
drh
d9c131c366 Add a test case to verify that the previous change avoids excess buffer
overread in sqlite3VdbeRecordCompare().

FossilOrigin-Name: 2b28e8d582cf10936fa1faca04a16ca2eeead66f
2014-03-26 15:05:56 +00:00
drh
182ea4e324 Earlier detection of corruption in sqlite3VdbeRecordCompare() in order to
avoid a potential buffer overread.

FossilOrigin-Name: 28ddecff044dbc2dff50a7d8406ab67dfe06587f
2014-03-26 14:51:07 +00:00
drh
e2ac506704 Add an assert() and a comment to clarify the operation of the
vdbeRecordCompareInt() routine.

FossilOrigin-Name: 851abdb8fd9b5a8a6ce21db53d30dbac3c430cc7
2014-03-26 12:02:38 +00:00
drh
5e8980ddb1 Adjust the repeat counts on several tests in speedtest1 so that the relative
time spent on the test is roughly the same regardless of --size.

FossilOrigin-Name: 7922809ee0d3978ce9221fbb7df63aa0684d0c4d
2014-03-25 20:28:38 +00:00
drh
e6c7b09df6 Merge enhancements and fixes from trunk.
FossilOrigin-Name: e005f2d6dd9faf38cc8fdb9428b5aa6192a6adae
2014-03-25 18:29:12 +00:00
drh
c872966ef4 Add another performance test case to speedtest1. This case is another
ORDER BY test but this time without LIMIT.

FossilOrigin-Name: 9ab7ffd59209aef0ffbf384b3902a93fd3b86a6d
2014-03-25 17:45:49 +00:00
dan
ab1dcc1a4b Fix a problem in the code added by [707ea170b3] causing vdbesort.c to sort unstably.
FossilOrigin-Name: d3e640afe611b6ae0b7f2cff5b00900d7e4d5ee3
2014-03-25 17:07:48 +00:00
dan
78d5843245 Remove the sequence values from sorter records used by ORDER BY as well.
FossilOrigin-Name: c3ae3697832a00d4d5758988a8766bdbb691e6b8
2014-03-25 15:04:07 +00:00
drh
0c60c1fe66 Add an ORDER BY test case to speedtest1.c
FossilOrigin-Name: 588122641e57e957813d329ea071e13ccbde5acd
2014-03-25 14:54:36 +00:00
drh
3c863634ef Enable four sorting threads by default in the command-line shell.
FossilOrigin-Name: 1cab83577c814feb35b4fb91af0d52a9751d99bc
2014-03-25 14:12:16 +00:00
drh
5c2b314222 Merge all fixes and enhancements from trunk.
FossilOrigin-Name: b415dfb6cb0df0c69992ca2bb700c15664f158e6
2014-03-25 13:17:41 +00:00
drh
86dd3716b9 Detect when a VdbeCursor is still pointing at a valid row but that row has
moved, and invalidated the return from prior sqlite3BtreeDataFetch() or
sqlite3BtreeKeyFetch() calls.

FossilOrigin-Name: e6798871ce94961135762669af418cd78540c121
2014-03-25 11:00:21 +00:00
dan
dd23c6bfb4 Omit the sequence value from sorter records used by GROUP BY queries that cannot use an index.
FossilOrigin-Name: 3f90abddc31ac20739778c235a834c33f7057997
2014-03-24 20:19:07 +00:00
drh
fd0a2f9756 Avoid unnecessary moving of content between registers during an ORDER BY.
FossilOrigin-Name: 4f472accf072d9cb64f209923924b26f21b13d27
2014-03-24 18:08:15 +00:00
dan
b22f38a779 Remove unused variables Parse.nColCache and Parse.iColCache.
FossilOrigin-Name: 4d7551ce464c8038147e81667368924f2a7485a6
2014-03-24 16:30:06 +00:00
drh
3d1d90a13d Fix arithmetic operators so that they do not change the affinity of their
input operands.  Ticket [a8a0d2996a].

FossilOrigin-Name: 221f8f944703108e47d789fa8ce6c00fe2abcbb6
2014-03-24 15:00:15 +00:00
dan
4979088506 Avoid attempting to mmap memory from an offset that is not a multiple of the system page size on systems with page sizes larger than 32KB.
FossilOrigin-Name: db7d62c8d58eb1e8654a762c9b199ae4e2759038
2014-03-24 11:23:17 +00:00
dan
3f802ebce2 Remove an unnecessary temporary register allocation.
FossilOrigin-Name: 5d506743f541b022cde04a9606baa4680cdfd70b
2014-03-24 09:34:58 +00:00
drh
83553eefaf Remove a pointless OP_Once operation in ORDER BY clauses with LIMIT.
FossilOrigin-Name: e6c59d23316c83b318b1a94d9b28a5d321737fa5
2014-03-24 02:20:53 +00:00