8743 Commits

Author SHA1 Message Date
dan
c578e4f0eb Fix a problem in the csv extension that was triggered when the very first field in the csv file is zero bytes in size.
FossilOrigin-Name: b12ddabf07b5e06bcee8dda3f990af3a131ab52b8fa969ba061b26d38458f31d
2022-07-05 17:49:04 +00:00
drh
0e4ab0db7e Enhance the REGEXP extension so that the end-of-input indicate ("$") is
allowed to occur on one branch of an OR ("|").
[forum:/forumpost/0107d5d40dd273e2|Forum post 0107d5d40dd273e2], second
issue.

FossilOrigin-Name: 3c04d21e6c632feb3bea8d1fa76bedcbfe254b0dc59865633d158a3f1bddefba
2022-07-03 18:12:43 +00:00
drh
f28727f61e Fix the initial-prefix optimization for the REGEXP extension such that it
works even if the prefix contains characters that require a 3-byte UTF8
encoding.  This should fix the problem reported by
[forum:/forumpost/96692f8ba5|forum post 96692f8ba5].

FossilOrigin-Name: c94595a6e15490b432f099fefbe2429fa19287f7bdc86332cba0fd1e08f65bd6
2022-07-03 14:25:47 +00:00
dan
2c1b1ddc55 Do not attempt the OP_Count optimization on queries with HAVING clauses.
FossilOrigin-Name: 566b7842ee286ed8620a87b06de65bc173ad0d208df4157292238fcb059cb484
2022-07-01 21:03:19 +00:00
dan
d85e4ac347 Add missing "finish_test" command to the end of test script merge1.test.
FossilOrigin-Name: 13cb3f1e63ed1e906f820655645a4966f0cae140ac442177b6685637dcfd365a
2022-06-30 11:01:48 +00:00
drh
fe9833482f Allow flattening of a subquery that is the right operand of a LEFT JOIN
in an aggregate query as long as there is no GROUP BY clause.  (The GROUP BY
clause will interfere with the operation of the TK_IF_NULL_ROW expression
nodes.)

FossilOrigin-Name: 2cf373b10c9bc4cbc5fe63d0a6948011df7bbc2f40dc025c9349f875da782b88
2022-06-25 19:43:44 +00:00
dan
a5ec23a705 In sqlite3WhereBegin, do not proceed with coding the loop if an error is detected as part of WHERE expression analysis.
FossilOrigin-Name: db5266dec601a9513bc8dd09a9f8bb4aef55b780d22610946099e8edd4836587
2022-06-24 11:05:36 +00:00
drh
2a40a882f2 A minor fix to test/fuzzinvariants.c so that it works even with column names
that originally contain a ':' and that are disambiguated.

FossilOrigin-Name: 8d9b1fff9d87522c4464aaf3ff0a7b93db244c59b4010562e35a8f161da4859b
2022-06-24 11:02:42 +00:00
drh
521e0b6cb6 New test cases for query flattening with LEFT JOIN.
FossilOrigin-Name: 27f68e47320c751e3663507500c1c44f0b7f885f89c678fce6a35b1bc372dd64
2022-06-23 22:43:11 +00:00
drh
d198183465 Add back the ability to flatten a LEFT JOIN subquery - previously removed
due to ticket [cad1ab4cb7b0fc344].

FossilOrigin-Name: f8fe936ad4f7678f9b26ba6fab41c5df9f4938634cdb3286b0f2dcd1357f919a
2022-06-23 15:15:03 +00:00
drh
f221e4b5d6 Fix requirements marks that changed due to typo fixes in the documentation.
FossilOrigin-Name: 5247df05991df979024df5a30bb3473401add92d4c12b01c040a16dda6da39f5
2022-06-22 18:33:21 +00:00
drh
b9294de1e6 Allow a HAVING clause on any aggregate query, even if there is no GROUP BY
clause.  This brings SQLite into closer agreement with PostgreSQL and fixes
the concern raised by 
[forum:/forumpost/1a7fea4651|forum post 1a7fea4651].

FossilOrigin-Name: 9322a7c21f1c22ba00e9b889223e89bc1591db6e561ce05091e905e98c1bf2b3
2022-06-21 13:41:24 +00:00
drh
436c28a265 Do not allow FROM-clause terms on the left side of a RIGHT or FULL JOIN to
be reordered. [forum:/forumpost/6650cd40b5634f35|forum post 6650cd40b5634f35].
This is probably more strict that necessary to get correct behavior,
but for the first release that supports RIGHT/FULL JOIN it is perhaps better
to be correct than fast.  A less strict constraint might be to prohibit
FROM-clause terms that originate on the left side of a RIGHT JOIN from
crossing from the right side to the left side of a LEFT JOIN.  Revisit this
later.

FossilOrigin-Name: 238453ffab0ba1bdddb529be35da82d5e8fb312a9574003a5441f455e601a909
2022-06-20 18:26:14 +00:00
drh
d7480403bc Do not allow an ON clause to references tables to its right if there is a
RIGHT or LEFT join anywhere in the query.  Other RDBMSes prohibit this always,
but SQLite must allow ON clauses to reference tables to their right for legacy
compatibility, unless there is a RIGHT or LEFT join someplace in the query,
in which case there is no legacy to support.

FossilOrigin-Name: e615dbe02ca949252d1526ed5c48f8ce08159773ea2008ce666484379d0d9854
2022-06-20 17:04:44 +00:00
drh
ae8776e092 The fix at [cab9b4cccd13bf0a] was incomplete, as demonstrated by
[forum:/forumpost/57bdf2217d|forum post 57bdf2217d].  This check-in
should complete the fix.

FossilOrigin-Name: fb0a23b6789da8e934562ce9ebd9d58ea13a10fd10dee5cbfc7ac8f394e1aeec
2022-06-20 12:42:28 +00:00
drh
13736999d9 Enable query invariant checking in fuzzcheck by default. There is no way
to turn it off.  Update the invariant checking logic to be consistant with
dbsqlfuzz.

FossilOrigin-Name: 66ca729bbbf37cb7ff8eb12f51429e0c0833bd5d3f0ef20a1eaeeb10820713c2
2022-06-18 20:20:30 +00:00
drh
d0d21f5565 Abandon a query-invariant check in fuzzcheck if the call to
sqlite3_bind_value() returns anything other than SQLITE_OK or SQLITE_RANGE.

FossilOrigin-Name: d31e1cd2ab44c7cce20b8990dff17719c286dd2fb46ba6d4f581a9553cf31891
2022-06-18 14:50:43 +00:00
drh
8fa6206bc4 In the --query-invariants option of fuzzcheck, correctly deal with OOMs
causing the return value of sqlite3_column_name() to be NULL.

FossilOrigin-Name: eabbee4a51bc1a865bddee890004ff3a1c9cc6b797b21f73e908642e154cef50
2022-06-18 10:26:12 +00:00
drh
c2beb0d8f6 Omit the --query-invariants processing in fuzzcheck for queries that contain
the implies_nonnull_row() test function.

FossilOrigin-Name: 0602a0844893465ac2fe53c0fa648a5cbcbc6452fff6181c42fab517f047b583
2022-06-17 17:11:51 +00:00
drh
f26e45139e Do not run --query-invariants on statements that make use of
sqlite_offset().

FossilOrigin-Name: c5ac4cbfb173c613d633b2c76484d735073692d001a336576174201b8f9af7b1
2022-06-17 16:52:54 +00:00
drh
ea64cb31a5 Allow integers and floating point numbers to compare equal.
FossilOrigin-Name: 0fe2e465ba931d4bceaf171693d2ee7af45a5f96840e65e9d4ee4a2e60f155d8
2022-06-17 16:32:21 +00:00
drh
63880362ed Improve query-invariants to the point that it would have discovered the
[f23a429d4153518d] bug if that bug had not already been fixed.

FossilOrigin-Name: 2a7251ed9c4c959b6df75172159c6b4445e1c2212e357d2c49ab8fa19cb3ac11
2022-06-17 16:09:47 +00:00
drh
c68fb84759 Only run query-invariant checks after the original query has run to completion,
so that we know it does not error-out on a subsequent row.

FossilOrigin-Name: d1fc857bb8dcd5914f5a9bbcc4efe5f4abe19a190e42e2c923b28327a95e4de6
2022-06-17 15:52:43 +00:00
drh
36f904f065 Fix the virtual table detection mechanism to avoid false-positives that were
blocking all failures.  Then fix a few of the additional problems that are
revealed by that fix.  More fixes are needed.

FossilOrigin-Name: 42b2e6676fed1508ea0ba17c292e83134825469735700da97817c45d45c54e66
2022-06-17 15:11:31 +00:00
drh
6efabd6cd3 Fix the new --query-invariants option on fuzzcheck so that it does not
use an unprotected sqlite3_value object as an argument to sqlite3_value_int64().

FossilOrigin-Name: d9f820151d74a690b5fa560597a5b3ace20165a112e1b58cb4a7c47b42745643
2022-06-17 12:25:33 +00:00
dan
2a7aff93ed Avoid omitting the rhs of FULL JOINs in cases where it is only correct to omit the rhs of a LEFT JOIN.
FossilOrigin-Name: f23a429d4153518d37387e121f22a30b22e2b31e126ad168e72049a96be86269
2022-06-17 11:39:24 +00:00
drh
3e245bcef3 Minor fix to the query invariant testing logic of fuzzcheck.
FossilOrigin-Name: 447e62a0946f5d77b7358adcabaeb23a7012cdfbfa1ef6082734cd9b45b2699d
2022-06-16 20:29:36 +00:00
drh
8f9261a8e9 Improvements to query invariants in fuzzcheck.
FossilOrigin-Name: 3a461f61b47e6ba6d5dcc2b7470ebde512b57bc68086f65050e07b06f42b7351
2022-06-15 20:18:44 +00:00
drh
e3bf2c8e9b Improvements to query invariant checking in fuzzcheck.
FossilOrigin-Name: 56c60a35ea457f06db58ec3f694a1ae16fd03e6625da1d7879d63d72bbcb1c62
2022-06-15 16:26:37 +00:00
drh
516c85026b Add the --query-invariants option to fuzzcheck. No changes to the SQLite core.
FossilOrigin-Name: fef282f9fbb10967102f9676d85b8c6a3b36aff7109f1becdaeaeddef0dff4f1
2022-06-15 12:50:51 +00:00
drh
0c278c3d76 Add the --query-invariants flag to fuzzcheck. The query invariant checks are
only run if that flag is enabled.

FossilOrigin-Name: d13b4621291831ff33cc4a8d53653eaa3f59fecf2fbc5d404db17e33a914495d
2022-06-15 10:46:52 +00:00
drh
a913f9b992 Improvements to query invariant testing. Almost working now.
FossilOrigin-Name: e039820418d64fb57cb1a8f9f21186284e6c76255a53445c5d7aef6cca89bfc4
2022-06-15 10:37:16 +00:00
drh
27a242c634 Fix the OP_NullRow opcode so that it works even if it applied to an
ephemeral cursor that has not yet been created.

FossilOrigin-Name: 0e925654c24774933c7738c68f704b229c68e18c8baa45f506b8d6f09164d0d8
2022-06-14 22:21:23 +00:00
drh
a1f79dae98 Attempt to enhance fuzzcheck to do some simple invariant testing on queries.
This is an incremental check-in for a work-in-progress.

FossilOrigin-Name: ce2d780163b3a28486904860a1815acc4169c09b971cfd199bb58d1e9a57b000
2022-06-14 19:12:25 +00:00
drh
92d1afba9b Do not remove the EP_CanBeNull flag from expressions during a LEFT JOIN
strength reduction if the query also contains a RIGHT JOIN. Fix for
the problem identified by
[forum/forumpost/b40696f50145d21c|forum post b40696f50145d21c].

FossilOrigin-Name: b1be2259e2e08ec22a88bc9a18b3ab4d83246ad4c635c05cdf80d3eff84df06a
2022-06-13 12:42:24 +00:00
drh
1cc5c4ac84 The same restrictions on the use of WHERE clause terms to drive indexes
in the presence of RIGHT JOINs also apply to the use of WHERE clause terms
to manufacture automatic indexes.  This fixes a problem identified by
[forum:/forumpost/51e6959f61|forum post 51e6959f61].

FossilOrigin-Name: 342c501f532523347e6c339351e02043dd6ee9e11a291224b65ea72bd6c2ba40
2022-06-10 16:41:54 +00:00
drh
958fcd417c Do not allow constant propagation between WHERE-clause terms and ON-clause terms
as this can confuse RIGHT JOIN.  Fix for the problem reported by
[forum:/forumpost/8e4c352937e82929|forum post 8e4c352937e82929].

FossilOrigin-Name: cab9b4cccd13bf0ab2bc38dc9a9c04ddd34e29c65ab6aef07b6bb3c31a43bece
2022-06-10 11:28:52 +00:00
drh
a27e350b6f Fix a faulty assert() statement identified by
[forum:/forumpost/0b91a75039|forum post 0b91a75039].

FossilOrigin-Name: 1f132bb03a22479ceeca0fd14940e2a1b29ce54b7784d5b0133450056813d877
2022-06-10 10:10:31 +00:00
drh
e5dea28482 Prevent subtype values from slipping across a subquery boundry when the
subquery is implemented as a co-routine.

FossilOrigin-Name: 9e51a6c0fbfb1899b2b01888430125fba6d4da9bad9eeaa3ad41e29fca54bbe5
2022-06-09 17:17:14 +00:00
drh
8878f8a8d3 The subtype of a value should not propagate across a subquery boundary.
Proposed fix for the problem reported by
[forum:/forumpost/3d9caa45cbe38c78|forum post 3d9caa45cbe38c78].
Additional works is needed as not all cases are covered.

FossilOrigin-Name: 08af1fe27ebd0edf6e0f1ac477deea033e7f7c813f1016b75196836daf02d2e4
2022-06-09 16:19:01 +00:00
dan
0a841a2b85 Avoid zeroing the value returned by sqlite3_changes() when a DML statement is automatically reprepared in sqlite3_step().
FossilOrigin-Name: 09c8f9f1970cd5b369d98a2b38f0b04d44ed095cb0bda80f7968bb6be4e0263b
2022-06-08 18:20:36 +00:00
drh
40357f0d5d Fix the query flattener so that it refuses a flattening that might leave both
an inner-join and outer-join ON-clause constraint (or equivalent) on the same
term of the FROM clause.

FossilOrigin-Name: f6c4fb48b65c2e8659aa0a1119c330e8bad5e42b2db2030850bfc9c04afef5c8
2022-06-08 15:38:16 +00:00
drh
a341bae948 New test cases refute check-in [3f45007d544e5f78].
FossilOrigin-Name: 12d3c96c39d12f6b01b77456eb874fff445ebfc60f50aada51b06ed37af364ea
2022-06-08 13:13:12 +00:00
drh
093dd41e97 Do not allow a partial index scan on the left table of a RIGHT JOIN, because
since the index is partial, some rows will be omitted from the scan, and
those rows will subsequently be picked up by the no-match logic in the
right-join post-processing loop.
[forum:/forumpost/c4676c4956|forum post c4676c4956].

FossilOrigin-Name: 615c0026119f7870c3b6ef9dcb57ce4ecf5acedea3e2b5cfc25aa450eb8f17a0
2022-06-08 12:46:58 +00:00
drh
f6bee94e22 Minor fix to test/index9.test so that it can be invoked with other
index tests using a wildcard.

FossilOrigin-Name: ccbd6e774903f9202f9311954979f6fcc3b13eba1f3402a549a7aee6ed1d55eb
2022-06-08 12:35:39 +00:00
drh
33b2cb9aec Fix the query flattener to deal with a RIGHT JOIN corner case described
by [forum:/forumpost/323f86cc30|forum post 323f86cc30].

FossilOrigin-Name: 3f45007d544e5f787d5837b4d9f484ba473d69cdba83c229228e9c2f6b972b75
2022-06-07 13:09:58 +00:00
drh
bb1369037b Test cases for [forum:/forumpost/323f86cc30|forum post 323f86cc30] added
to join8.test, though commented out so that they do not (yet) run.

FossilOrigin-Name: 7f97cb67d01a11f1b7e5b5f05824f9adbc6e4689d1809d60ceda50a595ba6d4a
2022-06-07 11:02:43 +00:00
dan
93c4087ffe Extra test cases for outer joins.
FossilOrigin-Name: c7505765280fb27ec2c8c4da6f87229a4078e3a4a27525226b6bf55864caa04b
2022-06-06 21:08:27 +00:00
drh
c18fc6134e Do not allow a WHERE clause constraint to be used to drive an index for
the right operand of a RIGHT JOIN, since this can cause problem if the
constraint implies a not-NULL value for one of the columns for the left
operand of the same join.  See
[forum:/forumpost/206d99a16dd9212f|forum post 206d99a16dd9212f].

FossilOrigin-Name: 4a31b7942a15c9c4363477365784d6d4ac5b1bbe8ff8aeaf2dd3d6532bf8bc96
2022-06-06 15:27:42 +00:00
dan
fd80fa9b6d Fix a test case in fts3corrupt6.test. Was failing due to [f1dfbc4f].
FossilOrigin-Name: 04b2e47784238008ce9057e2761ded91d895dc16d3bc8a3a3508df98a4769602
2022-06-06 06:17:54 +00:00