Commit Graph

19306 Commits

Author SHA1 Message Date
dan
c0bb4459d3 Fix another issue to do with window-functions in aggregate queries.
FossilOrigin-Name: 6413e38a174044c28fa9b8b937e6c972d144547a246e6f2882e782538300d042
2018-06-12 20:53:38 +00:00
dan
7392569f5e Fix some problems with using window-functions in aggregate queries.
FossilOrigin-Name: fe7081e0952950f577234fcbb58f3c1efa4579267654fd2f713dc4804e470e7e
2018-06-12 18:40:17 +00:00
dan
2a11bb23eb Clarify the relationship between a Window object and its associated Expr.
FossilOrigin-Name: 0cd55e98a478740032f5569ddc00fa5b0e063e90db6e00ac7598c9b7c2fffeee
2018-06-11 20:50:25 +00:00
dan
26522d1c45 Fix handling of window frames containing negative number of rows. e.g. "ROWS x
PRECEDING AND y PRECEDING" where (x<y).

FossilOrigin-Name: b6d9c7eda853420ae46a05bd432711e8bf9ebaa448c7d90ccfc0bcc338a87706
2018-06-11 18:16:51 +00:00
dan
72a9f0233b Fix a typon in main.mk.
FossilOrigin-Name: e74f86f271d6ab1ecd17c1ee63ab2aa0885ca56624be7382872f04d46c48ed86
2018-06-11 11:19:35 +00:00
dan
980cf25b10 Update Makefile.msc to include window.c in the build.
FossilOrigin-Name: 16db73842ade5eb0fe02f257b91d1c7b41d830d3f17f8638b8fbaed309d9a852
2018-06-10 07:42:35 +00:00
dan
96f55d0fa5 Merge recent trunk changes with this branch.
FossilOrigin-Name: c71f23590c25b4cecd27722e6c0fc8e3bf320d399c7d9398b7016dd5cf5b05eb
2018-06-09 18:09:44 +00:00
dan
d31e7addd6 Update the amalgamation build script to include window.c.
FossilOrigin-Name: 21d2f4a62eceab0de0b4669bf3e81eb71512211ed710ce0eb525695fb7309d6b
2018-06-09 17:58:51 +00:00
dan
8b98560d6e Add support for FILTER clause on aggregate window functions.
FossilOrigin-Name: ceaf798ea09184bc0e7d3dcf3ad4d909d2e4e7018754a8417a813f33010140a7
2018-06-09 17:43:45 +00:00
drh
4ece2f2657 Slightly smaller and faster code by encapsulating wal-index hash table
location information in a separate WalHashLoc object rather than passing
around the various elements as separate variables.

FossilOrigin-Name: 538a365b7a32ab7fa84f59d7556242cfb59b76d287b6417eb3a823197a354e8e
2018-06-09 16:49:00 +00:00
drh
a46ba6cca5 Improved comments an presentation for the recent IN operator decision
improvement.

FossilOrigin-Name: 31e480f68dfd887cfd9114f9f9fec53d751d3ecc27e36f55c0166b51b2fbb08c
2018-06-09 14:13:46 +00:00
drh
ccf6db5e64 Performance improvement to sqlite3WhereExprUsage().
FossilOrigin-Name: fd0934139076848f2f2edfd3d74d54608531031a05a60ca6ac1d7016dcd538df
2018-06-09 02:49:11 +00:00
drh
1fe3ac7347 Compute the bitmask of indexed columns for each index once when the Index
objecct is constructed, instead of recomputing it every time it is needed.

FossilOrigin-Name: d735872ec383bbd220b08c61d25db9ff3675d2542b9e7867e7d6323a12e0cc23
2018-06-09 01:12:08 +00:00
drh
da230bd484 Avoid invoking the whereLoopAddOr() routine in the query planner if there
are no OR operators in the WHERE clause, thus speeding up query planning
slightly.

FossilOrigin-Name: 292724ffc4bfca435fff309383d488ffdbe1e314e5eb26da21cf2f621b64bce5
2018-06-09 00:09:58 +00:00
drh
7128b7c1f4 When the query planner has the opportunity to use an IN operater constraint
on a term of an index other than the left-most term, use the estimated number
of elements on the right-hand side of the IN operator to determine if makes
sense to use the IN operator with index looks, or to just do a scan over the
range of the table identified by the index terms to the left.   Only do this
if sqlite_stat1 measurements are available as otherwise the performance
estimates will not be accurate enough to discern the best plan.  Bias the
decision slightly in favor of using index lookups on each element of the IN
operator.

FossilOrigin-Name: 2cbbabdf5ef624d809fbb40d2d312a29e0b5f02756fc0dbf6985fc8b0c8d1ade
2018-06-08 23:23:53 +00:00
drh
6d6decb8d9 Only choose to scan an IN operator rather than use an index if we have
real STAT1 data to suggest it is advantageous.

FossilOrigin-Name: 30e874661dcc1a2ecb40df2ef74582151d85bb36c754a38548829a3b6285f18d
2018-06-08 21:21:01 +00:00
dan
e3bf632cc2 Add support for the WINDOW clause.
FossilOrigin-Name: 19c983b511f1c823fdfb051713681b4c779f02fa83b41189afca0a9b8b72048d
2018-06-08 20:58:27 +00:00
drh
982c41977c Merge the btreeNext() assertion bug fix from trunk.
FossilOrigin-Name: 11bd66e09035039fbfc4c5631196962dd3f2e6634f5123a7b2fec17aeaaab07b
2018-06-08 19:54:07 +00:00
drh
f3cd0c82df Fix an assert() that can be false for a corrupt database and a strange query
that uses a recursive SQL function to delete content from a corrupt database
file while it is being queried.

FossilOrigin-Name: 99057383acc8f92093530e216c621d40386a06fe98131ff0af6df524d80a6410
2018-06-08 19:13:57 +00:00
drh
da4c409aea Consider doing a partial table scan to fulfill an IN operator rather
than using an index.  Try to pick the plan with the lowest cost.

FossilOrigin-Name: 1fa40a78fef4516c39b217bff67efe7e7d2077cca00aae0ef5c2c9cff94f008b
2018-06-08 18:22:10 +00:00
dan
dacf1de95c Do not flatten sub-queries that contain window functions.
FossilOrigin-Name: 236cb75bd1f0d5eb86aa5f52d8d548e7263c34633833dcea9dfc934f142113b8
2018-06-08 16:11:55 +00:00
dan
03854d2ecc Fixes to allow group_concat() to be used as a window function.
FossilOrigin-Name: 89bbc9ba8f66853a7530453f146c9df1baacd8558468016cefa7602911f7578a
2018-06-08 11:45:28 +00:00
dan
07509f8c1e Merge latest trunk changes with this branch.
FossilOrigin-Name: 251022034219819a1dc356542770ff46e3147a080f072eb20af6106771dadd92
2018-06-07 20:35:28 +00:00
dan
fe4e25a0f6 Add window functions lag() and lead().
FossilOrigin-Name: ef34207073c21ce8618486777671ae78d23f290acd6d3c37e91a49b69cb506ac
2018-06-07 20:08:59 +00:00
drh
83193d0133 The IN-early-out optimization: When doing a look-up on a multi-column index
and an IN operator is used on a column other than the left-most column, then
if no rows match against the first IN value, check to make sure there exist
rows that match the columns to the right before continuing with the next IN
value.

FossilOrigin-Name: 09fffbdf9f2f6ce31a22d5a6df7a45f19a16628da622f12d6e33171cce09fb21
2018-06-07 18:13:49 +00:00
drh
cf025a8c18 Fix the assert()s in the byte-code engine that prove that cursors
are unidirectional.

FossilOrigin-Name: 4b0b4e14039469b656662312a5f80f086ede293e9ad04c7bc99a202b683a1e55
2018-06-07 18:01:21 +00:00
dan
7095c002b8 Add support for window function first_value().
FossilOrigin-Name: 060b26402880daab085ad01f5f0dbde957c7a22cd219be5b8ec94fba883051a0
2018-06-07 17:45:22 +00:00
drh
f1949b6634 Remove the NextIfOpen and PrevIfOpen opcodes which are no longer needed
when the IN-early-out optimization is working.

FossilOrigin-Name: 439c8162272795b422a0e01b01b832fbc12b39914c9632a674162af8bdecff98
2018-06-07 17:32:59 +00:00
drh
056f5396fe Test cases.
FossilOrigin-Name: 085e863713a3f2d420c0076b275a6ac445a59d4d93f9eb0e8503b4e3f5589249
2018-06-07 16:07:00 +00:00
dan
2e60568fef Fix problems with the nth_value() function.
FossilOrigin-Name: 63002b9a090397668b94fe341a035c42f84e4837af85395bce4cd1c2777c5209
2018-06-07 15:54:26 +00:00
drh
64c1f77a8e Merge the ".stat/.eqp" CLI fix from trunk.
FossilOrigin-Name: a91cad3381bb843d6f58975251bf99f0fa1a1398fae53d97a98a6c8ee65e718e
2018-06-07 15:28:40 +00:00
drh
3c49eaf4be Avoid using a prepared statement for ".stats on" after it has been closed
by the ".eqp full" logic.  Fix for ticket [7be932dfa60a8a6b3b26bcf76].

FossilOrigin-Name: bb87c054b1b76959e46258ac66b24027f468b390a4148ac67f208a1fbeda4060
2018-06-07 15:23:43 +00:00
drh
f7b0a5f3c3 Add the WHERE_IN_EARLYOUT flag and use it to clarify the logic of this
optimization.

FossilOrigin-Name: 522f1eacc20f11002cad58232a7c2610f369568653510e54f46088f579f778dc
2018-06-07 14:59:22 +00:00
drh
bd573c459b Merge recent trunk enhancements.
FossilOrigin-Name: e9d7bf4f7b9d6f8dabc4c95d43ebf12f2149bed1c5e750048b1b684128073c38
2018-06-07 14:32:16 +00:00
drh
df12595c09 Change a comma into a logically equivalent but semantically clearer semicolon.
FossilOrigin-Name: 71f97f0f82b3abfb07feb78d64a182fc50ff396e85d6f5aac479dbf58ba4d00a
2018-06-06 23:31:26 +00:00
drh
6eb3480bd7 More space and performance enhancements to sqlite3VdbeRecordCompare().
FossilOrigin-Name: 83a60ff056a63f18479030e9dfb10926fbb0d906d51f2cf88233098e15c75534
2018-06-06 20:55:10 +00:00
dan
ec891fd40c Add implementation of nth_value() window function.
FossilOrigin-Name: eb1fb420ac70857e7ca4cbad78be15a27bee8f915ba2da080c7efa1879c06c31
2018-06-06 20:51:02 +00:00
drh
d882108a01 Small performance optimization in sqlite3VdbeRecordCompareWithSkip() for
the common case where the comparison is equal.

FossilOrigin-Name: 1e616e256a4fb1b64271706fdfa77dc5790eba0a2f0e619544e169bc61d7c805
2018-06-06 20:29:19 +00:00
drh
4011c44322 Rearrange the order of some checks in the integrity_check pragma for a very
slight performance gain.

FossilOrigin-Name: 4b853f020570bf4af1b14e03f35764c7d7a03a40af58efc783e06f2b883cef78
2018-06-06 19:48:19 +00:00
drh
685a50ad74 Fix the sqlite3BeginTrans() calls within the snapshot extension.
FossilOrigin-Name: 1fef7ad25b6a8e59163ac57f5cf0412bfc5b0e9446fd782f5f0a7d7ee11741fc
2018-06-06 18:50:50 +00:00
drh
397776a823 Another minor optimization to OP_Transaction.
FossilOrigin-Name: d80077aee3904e5d93164b342cae14d813de8e84e567462412751e06c7487d41
2018-06-06 17:45:51 +00:00
drh
bb2d9b1bf1 Change sqlite3BtreeBeginTrans() to return the BTREE_SCHEMA_COOKIE, for a
small speed improvement when starting new transactions.

FossilOrigin-Name: a10662aa915ae2b5a78b3e10920350d32255b2d6b1a8aac0aba1ad173b07ed2b
2018-06-06 16:28:40 +00:00
drh
0cfd46a1ad Fix the ".archive" command in the CLI (and the corresponding -A command-line
option) so that it silently ignores filenames that contain "../" in their
names.  This prevents the "Zip Slip" attack.

FossilOrigin-Name: 27291f2d7fd4dadf2ee9b9a7d1373158cadfbaf83c3654b00d7030dc921770c6
2018-06-06 01:18:01 +00:00
drh
bb777d517d Update the version number to 3.25.0 for the next development cycle.
FossilOrigin-Name: 7598236c356cdb548c6188d69dfef99f7a08b89e512a3addfe1433ccd85e7b68
2018-06-05 23:51:52 +00:00
drh
48baed4f77 Avoid unnecessary loads of columns in an aggregate query that are not
within an aggregate function and that are not part of the GROUP BY clause.

FossilOrigin-Name: e15e100660d290249ef235e7a8927b88296e56ec0f80ec626eecbd542adc7633
2018-06-05 23:21:11 +00:00
drh
8c2b6d784b Add the OP_IfNoHope and OP_SeekHit opcodes used to reduce the number of
unnecessary sqlite3BtreeMovetoUnpacked() calls when checking for an early
exit on IN-operator loops.  Futher optimizations are likely possible here.

FossilOrigin-Name: 6bf251af4347165a470d39457d61ab6d2a06c206db8f30bd8be5dbb388ae8a5b
2018-06-05 20:45:20 +00:00
dan
280c894b74 Calculate non-aggregate expressions in the SELECT list of an aggregate query
that does not use min() or max() once per group, instead of once per row
visited.

FossilOrigin-Name: dce2dfbe1590deb3ef5661230ae2d232bd492441195defbf698ac56f9629211c
2018-06-05 20:04:28 +00:00
dan
1c5ed624a0 Add implementation of last_value() window function.
FossilOrigin-Name: 2493ce1a402888f6a8f25e03052c25a4e9bca740639e98cb614f5e5160ac25d3
2018-06-05 16:16:17 +00:00
drh
86d0ea7558 Use an OP_NotFound opcode to cancel futile IN operators early. The current
implementation is suboptimal because it always runs teh OP_NotFound.  This
still needs to be enhanced to only do the OP_NotFound if no results have been
seen on the current loop.

FossilOrigin-Name: 87a9fc504f9a78caf7a7949cc7ada0a19d61bfab51bb49a00a1607194c116212
2018-06-05 15:16:25 +00:00
drh
995bbbf993 Merge trunk changes.
FossilOrigin-Name: 047295c588e9fdf2ffa4e69e166f177fd193309531dc6a9ac755fb7a763adb72
2018-06-05 13:54:06 +00:00