Commit Graph

6828 Commits

Author SHA1 Message Date
dan cfaffad65c Fix a test case in zipfile.test.
FossilOrigin-Name: 4eb5b24c64ee5907c18371f563b79fa0caf58285a62b8f09e639a690e6727eaa
2018-01-31 19:45:58 +00:00
dan 44091ed368 When creating a new archive entry, have zipfile store UTC instead of local
time in the legacy MS-DOS format timestamp field.

FossilOrigin-Name: e2114df18383d111dd5fbac902e08b42a7f4b2b2d6f7bf29574a3722e4a4dad5
2018-01-31 19:13:31 +00:00
drh 5cd253423b Improve the omit-left-join optimization so that it works in some cases
when the RHS is subject to a UNIQUE but not NOT NULL constraint.

FossilOrigin-Name: 02ba8a7ba7ba71cd7abd5dd3093ea486f53a025f6972bb444f2da37e0e2fc3b2
2018-01-31 16:50:27 +00:00
dan 4ea48144f9 Fix a failing assert() in the new code on this branch.
FossilOrigin-Name: 74d857d178dfadea7e07ba47439fe11aa9d282b54caf78cd6961e593b37406d0
2018-01-31 14:07:01 +00:00
dan 099fa84727 If a virtual table xColumn method sets an error message using
sqlite3_result_error(), use that message in preference to any error message
left in the sqlite3_vtab object.

FossilOrigin-Name: 71e3b715ad36c2132a587b84221be6359c31d7a17c3fba201b7ed3baf17922f1
2018-01-30 18:33:23 +00:00
dan 2f7260deba Add tests cases for the zipfile extension.
FossilOrigin-Name: 13b786dafd308b7aef42328f26fe64981c83d524e4bf82ceb7d58afc7146c7d3
2018-01-30 17:43:22 +00:00
dan fdcd9d4e6c Prevent users from creating zipfile() virtual tables without an argument.
FossilOrigin-Name: 81fdbe0cc5a360f818078d47a5888d0a29d555927da279a9a0213702d74ef09a
2018-01-30 14:07:55 +00:00
dan ea4125bbef Add unix-only tests to check that the "unzip" program can unpack archives
generated by the zipfile extension.

FossilOrigin-Name: 438c5c5237a801ae78809bf324bb9251fb50250addfc1f8e36659442b0e26ab6
2018-01-29 19:47:32 +00:00
dan f8c4b99a02 Add aggregate function zipfile() to the zipfile extension. For composing new
zip archives in memory.

FossilOrigin-Name: e364eeac76a8225146b37d801bc6cabe03e9abede5a1412ebe9d94a32d8838cc
2018-01-29 18:41:07 +00:00
dan 8433e7166b Update the omit-left-join optimization so that it works in some cases when the
RHS is subject to a UNIQUE but not NOT NULL constraint.

FossilOrigin-Name: 88411a402b29f19d69e0d06d9099eabb98c238ddb7c07ce110236316f008753e
2018-01-29 17:08:52 +00:00
dan 26e731cc88 Ensure the "unique-not-null" flag is set for automatic indexes on columns
declared with "col UNIQUE NOT NULL" (where the NOT NULL comes after the
UNIQUE).

FossilOrigin-Name: 8767f7b880f2e4112f75f0b6ef7be3f50ab1ae20e103e7d03d8bfe77e6c79438
2018-01-29 16:22:39 +00:00
dan 26333ee3ca If a zipfile virtual table is created with no argument - "CREATE VIRTUAL TABLE
zzz USING zipfile()" - accumulate data in memory. Support "SELECT
zipfile_blob(z) FROM zzz LIMIT 1" to retrieve a zip archive image.

FossilOrigin-Name: e63185edfe0c316aa60c1fa085d032425ecc7db54536dfa5a977772eaf3c240e
2018-01-27 16:29:59 +00:00
drh c9f3db33d5 New test case for ticket [ec32177c99ccac2b1] that works without the STAT4.
FossilOrigin-Name: 5259d4847f2b73f26b2385f9d8cff8fe0cabc54b4deab8477c87c8d1bb5535b1
2018-01-27 13:55:56 +00:00
drh 33f10207aa Proposed fix for the query planner problem of ticket
[ec32177c99ccac2b1].

FossilOrigin-Name: eef8cbef3c49e6331301a8f3f8a5fd12090b1e99e68487c2d405e53ef771f688
2018-01-27 05:40:10 +00:00
drh 33f763d154 Fix the query planner so that it takes into account dependencies in the
arguments to table-valued functions in subexpressions in the WHERE clause.
Fix for ticket [80177f0c226ff54f6dd].

FossilOrigin-Name: 7daa687340e475972d6849f84fe1419a1f5700934f53fd80656849adc73d6d04
2018-01-26 22:41:59 +00:00
dan 8005d60587 If the argument to table function zipfile() is a blob (not text), assume that
it contains a zip file image to interpret, not the name of a file on disk.

FossilOrigin-Name: 029ebcd30cb261d949f7587ac54c23d7479796b6716fd4ca7512361b8f32de3e
2018-01-26 18:59:25 +00:00
drh 4eb57ceffc Improve text-to-integer conversion in boundary cases. The
sqlite3Atoi64() function always returns the minimum or maximum integer
if the magnitude of the text value is too large.  Trailing whitespace is
now ignored.

FossilOrigin-Name: ace0644a1a2a42a3ea42d44f00a31915b8a7e56c9ba90f90a6c02001f89f9c86
2018-01-26 18:37:34 +00:00
drh fe875027d7 Add the normalize.c extension.
FossilOrigin-Name: 16ebe558d2f0982f630c39b6c1b00d14d27011d454cced213f17c26491a2c06e
2018-01-24 15:07:37 +00:00
drh 31a13548f8 Merge changes from trunk.
FossilOrigin-Name: 6ef3de810d54563c227045b16197b8011ce285ea867261946f93b2de55344f29
2018-01-24 15:02:53 +00:00
drh 4d6d872c49 Adjust the fts3rank.test module so that it works on big-endian systems
(hopefully - I don't have a big-endian machine to test the change on.)

FossilOrigin-Name: e4766cabdf64d8e998048ae43154466fc9c3fad0b86102a42b65122abfcbbe55
2018-01-24 14:39:18 +00:00
dan 0aac5613d9 Update a couple of test scripts so that they work on F2FS file-systems that
support atomic transactions.

FossilOrigin-Name: 49e58e645e0c114c71935a3b7fa4771e8a23e28127efd2efe0f2f18813c8f391
2018-01-24 06:30:10 +00:00
drh 56f1774603 Make the shell functional even if compiled with SQLITE_OMIT_COMPLETE.
Omit the sqlite3_complete() call from the fuzzing interface if it is
compiled with SQLITE_OMIT_COMPLETE.

FossilOrigin-Name: c3e816cca4ddf0967c4c790cdde2345101dde3f2d854e62589f1ac1d7f3c60b8
2018-01-24 01:58:49 +00:00
drh 5347f3c111 Invoke the sqlite3_complete() interface from the fuzzer.
FossilOrigin-Name: 332bf84625d2034c9e1c029aa8243aa34088217e52e34e1f5472217743a31af3
2018-01-24 01:02:23 +00:00
dan a916b57013 Fix a bug causing spurious "sub-select returns N columns expected 1" errors
in join queries with a term like "(a, b) IN (SELECT ...)" in the WHERE clause.

FossilOrigin-Name: 14dfd96f9bca2df5033b2d894bf63cc8bf450a45ca11df5e3bbb814fdf96b656
2018-01-23 16:38:57 +00:00
dan b8fff29c68 In SQLITE_ENABLE_BATCH_ATOMIC_WRITE builds on F2FS file-systems, invoke
SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE if an SQLITE_FCNTL_COMMIT_ATOMIC_WRITE call
fails. Also, do not use an atomic transaction to create the initial database.
This is because if an error occurs while writing to the db file, any changes
to the file-size do not seem to be rolled back automatically. The only time
this matters is when the file was 0 bytes in size to start with.

FossilOrigin-Name: b3122db1545aeb48b7c28d480534b4b0fe04e83d5336225714c3cad926e5960e
2018-01-23 14:01:51 +00:00
dan 3a9b9b7749 Fix a test case in walro2.test so that it works with encryption extensions
(when each page has a little less usable space available than usual).

FossilOrigin-Name: b79521d255a4aa2a7b4424f3498443f27d329ab5428171ba6b87c987f3ccdfc2
2018-01-22 15:20:59 +00:00
drh 88284b0dc3 On the walro2-X.4.2.2 test case, show the size as part of the test output.
FossilOrigin-Name: ac5091d7f870f1d0659a576723de454d544e5bfa2bdbba307d7c81eb5ab34e3b
2018-01-22 14:49:15 +00:00
dan f193937620 Omit the single test from zipfile.test that uses json functionality in
non-SQLITE_ENABLE_JSON1 builds.

FossilOrigin-Name: 6bb2a10fad71bf8cec6bca538db7b00be0d26418ab62b83092d3a6a68d1c0d9f
2018-01-17 12:58:11 +00:00
mistachkin 824f28426a Fix duplicate test names in 'walfault.test'.
FossilOrigin-Name: 7274d05ff43fc9872f0e4857ae583689e4a6c429b7fa991dcc29744da3048879
2018-01-17 01:15:08 +00:00
dan 2879952faa Fix a problem causing zipfile to store 0 in place of the CRC32 value for
uncompressed files.

FossilOrigin-Name: ba44724bcca2e87788b7c6d8c5de7fa388360127bd894ee6a171fd66e794fcae
2018-01-16 21:09:00 +00:00
dan 0c2ba13e02 Fix a problem causing an infinite loop or other malfunction in some UPDATE
statements with an OR term in the WHERE clause.

FossilOrigin-Name: feb2c2b6f66b0f45490beb1642d99cdb89fa220e299a8c118929df557c814189
2018-01-16 13:37:43 +00:00
drh ded2d99d73 Disable the ".archive" command tests in shell8.test if the CLI is compiled without
ZLIB support.

FossilOrigin-Name: ce8bfe6c2b87090a2de1e04bc88fcb878597fe1f4ecd5df6d9d588a65601c901
2018-01-16 02:38:35 +00:00
dan 2d620070b9 Fix a problem in the zipfile module causing it to generate incorrect
checksums. Remove the ability to insert compressed data into a zip archive.

FossilOrigin-Name: b0b7d0363acf38c2178e2d3041d8ce2a0de061a51caa64670dbf539ee6d4356b
2018-01-15 19:00:35 +00:00
drh b058d05452 Avoid excess stack usage when a VALUES clause with lots of rows occurs
within a scalar expression.  This fixes a problem discovered by OSSFuzz.

FossilOrigin-Name: a4fa0581ba7cfd45fabe0198f55b3c2c8ee3ecfd2825aeed91116f44e77d760b
2018-01-14 20:12:23 +00:00
dan f42884c307 Support UPDATE statements against zipfile virtual tables.
FossilOrigin-Name: f2d2a5df4f29b47212fd2411eae6545087b901a270655640c87ceb472e02a24c
2018-01-13 19:08:24 +00:00
dan 2483a111db Fix a typo in crash8.test.
FossilOrigin-Name: c3dc7b8d9de94e46fae75fcc96ec68743205e0a2591e14391c394fff203004d1
2018-01-13 14:01:26 +00:00
dan 69aedc8db4 Fix various problems in test scripts preventing "make test" from passing on
F2FS file-systems with the "atomic-write" feature.

FossilOrigin-Name: 56d93d070d6b92d8a5a3fec1b09aae8911116c73d072fc5022f0b51668ed996b
2018-01-13 13:07:49 +00:00
dan 614efe2b4b Experimental change to include changes made to the sqlite_stat1 table in
changesets generated by the sessions module. sqlite_stat1 entries in such
changesets are ignored by legacy clients.

FossilOrigin-Name: bd46c4429693545eb16db85692fc591ac529796aa746f5f21df1ce4380619320
2018-01-12 16:44:29 +00:00
dan 66a3a91aa6 Change zipfile to be a WITHOUT ROWID virtual table and table-valued function.
FossilOrigin-Name: 931201f64e04247ed613a0301fcc86c3a337c2ed162c6370a80c67a1dd919e7c
2018-01-11 17:33:48 +00:00
drh 6c9dcc427d Update test cases for the new "usage" for .output.
FossilOrigin-Name: fbf5e43c07e7c012cb39b33a74b3fab9e46ba946c48497fbd990110692125f57
2018-01-10 23:27:30 +00:00
drh e15c5e532a Add the lsmode(MODE) function in the fileio.c extension and use that function
in the .archive implementation.  Add the --append and --dryrun options to
the .archive command and remove the --zip option, making it automatic.

FossilOrigin-Name: 38f28029d16df8489772c27867de8888dd1c26b170861acdc977fd4023939dde
2018-01-10 17:44:03 +00:00
dan 4bfd182939 Return an SQLITE_CONSTRAINT error if an attempt is made to insert duplicate
entries into a zip archive.

FossilOrigin-Name: 1f099b2b45074c89eeff8ff241aa49489c95c2221b25c305fcda670ebf63fb4e
2018-01-10 17:37:58 +00:00
dan f2ed70e4de Have the zipfile module automatically append "/" to directory names that do
not already end with such a character. This is required for info-zip
compatibility.

FossilOrigin-Name: 94bc3c60e7d2ec849b90444b06e3057ed645edf3af334f2737252960602868e5
2018-01-10 16:30:40 +00:00
drh a5676c4d2d The ".ar" command deduces whether or not the target file is
a ZIP or SQLAR and does the appropropriate thing.  The "-z" option is omitted.
The "--append" option is added to open auxiliary databases using apndvfs.

FossilOrigin-Name: 430d1a7daa823ae53606b7a158af4e7c16f62ff9b072b90606524e7c3f6131df
2018-01-10 15:17:34 +00:00
dan a688ca5ea9 Fix a problem in os_unix.c causing it to return SQLITE_CANTOPEN instead of
SQLITE_READONLY_RECOVERY.

FossilOrigin-Name: 6a16f554f027ba268276b728588b5eaea837cbed85358a06a2f6da3b70e834ad
2018-01-10 11:56:03 +00:00
drh 6116ee4eee Compute the correct column name even if the column identifier is the
very last token in the SQL statement.  This fixes a problem introduced
by check-in [0fdf97efe5df745510c6b] and reported by the community during
beta-testing.

FossilOrigin-Name: 36b89d728ff13d395fe0e1db8e7c01263f73dccb278b3ece27f6ef78e909b492
2018-01-10 00:40:06 +00:00
mistachkin dc006e05aa When disconnecting from the 'swarmvtab' extension, close each database prior to invoking the 'openclose' function on it.
FossilOrigin-Name: 3e5647cb6c4553683e24b9cb62548f16c79c4e2ac9e39cf135ea52a623f7cc33
2018-01-09 22:23:42 +00:00
dan 2c4df97591 Return SQLITE_CONSTRAINT if a user attempts to update a zipfile table.
FossilOrigin-Name: 64c9ccf6c5598a68135e78dad128ec070afa49416e1d1f35745a24276e7aeeea
2018-01-09 20:44:04 +00:00
dan c48e0271f6 Fix a potential crash in sqlite3expert.c caused by a missing sqlite3_reset()
call.

FossilOrigin-Name: f6355970f04a3c4d85801c806366788139d8d5ad29383ec7faa81e2176ffe987
2018-01-09 18:31:11 +00:00
dan 128011a258 Fix a zipfile problem with adding new directories to an archive.
FossilOrigin-Name: 5fed67033c9dd4492bf8cfcf98874284581f448d8cc84fa5470dde239f218375
2018-01-09 10:29:34 +00:00
drh 91694dbdd2 Test cases for sqlite3_normalize().
FossilOrigin-Name: 658f42257d56a3562dfa8e55023c6e497f55c565cd751d942de3a9e0cb2fa708
2018-01-08 20:04:57 +00:00
dan 7c15ac1ada Update the zipfile module so that it matches the documentation.
FossilOrigin-Name: 7e7e472fa91a2bad2e521d4d67f176c8eb9edc1a07b283e425ea0fa2b6abba1f
2018-01-08 19:59:59 +00:00
dan 4064a4cc58 Add an "ifcapable trace {}" block to a test in fkey1.test that uses SQL
tracing.

FossilOrigin-Name: 7182591d351dde22ed2f6a60521d1d7c10a610d702e79693412efc6938167be0
2018-01-08 17:12:54 +00:00
mistachkin f2e8aa637e Skip running the 'zipfile' test if the necessary static package cannot be loaded.
FossilOrigin-Name: 5bc816ec4010c20407396a1051e9f9b0a360ca27c0a99dafa867225cf974e306
2018-01-05 20:13:33 +00:00
mistachkin 992fdfc5d8 The 'releasetest' tool should pass the TCLDIR macro for MSVC to nmake.
FossilOrigin-Name: eaa9c0dda73808f1458f9968be59947636fbd4781c9fc535eb50bcaf790093c1
2018-01-05 19:27:19 +00:00
mistachkin dfdfd8c7f2 Get the shell enhancements compiling with MSVC.
FossilOrigin-Name: 335387f9e0d4569097d34cd99cd332b38a282e9b7ae25f088eb47df5c25837ef
2018-01-04 22:46:08 +00:00
drh c76ed3d57d Merge in all recent trunk enhancements.
FossilOrigin-Name: 406f79183736b6ad360169b837172afef2c82a4312f5787db08c54167a44b15e
2018-01-04 19:54:55 +00:00
drh 35100fb194 Remove the snarky "_supported_" qualifier from the name of the
sqlite_offset() SQL function.

FossilOrigin-Name: a6eee0fcd89d3958f8720ebdb5f0a8558b4795d747128091dae283eb81c4f74f
2018-01-04 19:20:37 +00:00
drh a5bb43518b New test cases and fixes for minor problems.
FossilOrigin-Name: e0da2cc382e0db33d0423372133115f52c77bd0093dbf816c7e88a0aad5f6582
2018-01-03 23:40:02 +00:00
drh 3ec8665e22 Replace sqlite3_memdb_config() with sqlite3_deserialize(). Remove the
"db memdb" command from the TCL interface, replacing it with "db serialize"
and "db deserialize".

FossilOrigin-Name: 2f6e9df9f0c5a9e5b1acb99cfa9486850cc1822d35b0989e779a7a10f3b1f1ac
2018-01-03 19:03:31 +00:00
drh cb7d541d3a Replace the sqlite3_memdb_ptr() interface with the more general
sqlite3_serialize() interface.

FossilOrigin-Name: 8cf2ed4eff6d2e0958656e23384b05ead2128b678b0b69a591878af4190cd077
2018-01-03 16:49:52 +00:00
drh 4dcac40e3d Simplify the "sqlite3" command in the TCL interface. The filename is now
optional.  There is a new --memdb option with an argument that is the blob
to which the database content should be initialized.

FossilOrigin-Name: 47398ae77236a92f6b9345aa397361b6df127a9a2895c0771d506b0be10822b9
2018-01-03 13:20:02 +00:00
drh 9eb8dbad4d Add another test case (found by OSSFuzz) for the problem fixed in
check-in [2846458a] and described by ticket [dc3f932f5a147771].
No changes to code.

FossilOrigin-Name: 4165fae920a1176c68bfaa1f2cfe17df7f8dd8793519d458b71e3e63670b5495
2018-01-03 01:47:30 +00:00
drh ac442f41c1 Add support for the "memdb" VFS and the sqlite3_memdb_ptr() and
sqlite3_memdb_config() interfaces, to enable an SQLite database to be
manipulated as an in-memory object.

FossilOrigin-Name: fb2ac2d2fa6374084f3325b41b257c7a3ace43aade4b666ec4be93b6b70dc39a
2018-01-03 01:28:46 +00:00
drh 4d795ef7e4 In the constraint resolution logic, be careful not to cache column values
in registers whose initialization might be bypassed by an OP_NoConflict opcode.
Fix for ticket [dc3f932f5a147771] reported by OSSFuzz.

FossilOrigin-Name: 2846458af5d029a8e4fdcc8f50873a44e57897bbfe6aee8a23a01ffc34c5579f
2018-01-02 18:11:11 +00:00
drh 1d315cf7b8 Improved name quoting and escaping in the auxiliary column info section
of the ".schema" output for views and virtual tables.

FossilOrigin-Name: d64b14e37d9624bf5d86059ddd091170d8e6d341a8043f84548b9b3dbb96a908
2018-01-01 21:49:43 +00:00
drh ceba792a2f In the output of ".schema", show the column names of virtual tables and
views in a separate comment.

FossilOrigin-Name: 2234a87fa905312b23f46d52e06cff7cacbf23b187e16c4398a42e6bdae0ee9f
2018-01-01 21:28:25 +00:00
dan 0cde0c62b1 Have zipfile support DELETE commands.
FossilOrigin-Name: 01d4e866fb7b01aeada537d41c4a47747c7810e2028f51077ee5b8b78c348954
2017-12-30 18:32:27 +00:00
dan db0cb303ad Rearrange things a bit so that writing to a zipfile does not invert the order
of objects it contains.

FossilOrigin-Name: f69e8194bfa7de436c96028730ebd57f186d2e6207792e172e1aa38c7f4211c9
2017-12-30 14:26:29 +00:00
dan 373dc3bb3f Update ext/misc/zipfile.c to support creating and adding entries to existing
zip archives.

FossilOrigin-Name: 2dec2dec592c0726ebe87b841b9c8d493dea7074a99f278eb1bf0b744d658a9d
2017-12-29 20:19:03 +00:00
drh 9af41ff452 Stricter test cases.
FossilOrigin-Name: 9406c0a685fd5ff3516a66402b0514a1440652822a5eecf0b7af85929f3079e8
2017-12-29 16:37:33 +00:00
drh 092457b18c Change the function name to sqlite_unsupported_offset(X). Only enable the
function if compiled with -DSQLITE_ENABLE_OFFSET_SQL_FUNC.  The makefiles add
that definition to shell builds.

FossilOrigin-Name: 7a7f826e324b1a2c332e2f1d0740fd0babffcaca6275a798572f02ad367b99ab
2017-12-29 15:04:49 +00:00
drh fe6d20e9f4 Enhance location(X) so that it works with indexes and WITHOUT ROWID tables.
The function might return an offset to the main table or to an index,
depending on whether the column X would be loaded from the main table or
from the index.

FossilOrigin-Name: dd94d6a880dfec4bddd247239b815b84964f804d24841e25f33f1d46a4b5274d
2017-12-29 14:33:54 +00:00
drh 7a8573bafa Merge recent enhancements from trunk.
FossilOrigin-Name: 6251e438f2a76170fd1e95aa512a46086ed88ab93b9b97a1dba97c4558689305
2017-12-29 13:35:09 +00:00
drh ee052a1c94 Add test cases for the undocumented behavior of duplicate columns on an
INSERT or UPDATE.

FossilOrigin-Name: f4349c0c26611de8a7d5beb99431a575cf531cdeb0ca2413efabcf0a61e6f424
2017-12-29 12:50:43 +00:00
drh e307e11d0a Omit all sqlite3_trace() output from the triggers associated with
foreign key constraints.

FossilOrigin-Name: fda08e3d10cc850664a356efdafcfc68187053849e1b00991b0b35d892a6776b
2017-12-27 21:30:34 +00:00
drh f259df5fc8 Show the text of individual statements within a trigger, as they
execute, as comments in the output from sqlite3_trace() and sqlite3_trace_v2().

FossilOrigin-Name: fe3d2b97d8945e6a9636472f77bcdc92cc5dda8b3e6e3a4fcbdd0a212ea5b9aa
2017-12-27 20:38:35 +00:00
drh 9b2e0435d2 Add and use the internal sqlite3DbSpanDup() interface.
FossilOrigin-Name: a8e1545cb7aacb6a26a8c92a3ad4a3d584d150c3a00d2828c8adbb1ee19fcb6d
2017-12-27 19:43:22 +00:00
dan f6c37dbc8c Add options to speedtest1.c and speed-check.sh for testing performance of
floating-point computatations.

FossilOrigin-Name: ebfea8728fec955b1d74b1d0a3de498fd1a32e8b39572a8fdab606ed87b169b4
2017-12-26 14:30:44 +00:00
drh 5060a67c4d Fix an assertion fault on a syntax error input caused by check-in
[6b2ff26c25bb9da3].  Problem discovered by OSSFuzz.

FossilOrigin-Name: 90d6e4f10d3055a776d24854c442a2a68e726af8fc382cdb6241a834082e4c4c
2017-12-25 13:43:54 +00:00
drh 512795dfea Fix a NULL pointer dereference after a syntax error that can occur as a
result of check-in [6b2ff26c25bb9da3] yesterday.  This problem was
discovered by the OSSFuzz.

FossilOrigin-Name: d49afb8f9804e96662d1e3cadc4c6643908706d848a53d5ed019919c98f2ccba
2017-12-24 18:56:28 +00:00
drh 60ba57d9c5 Merge enhancements from trunk.
FossilOrigin-Name: 150f07fec1e6d1fc0601820d717d8712fc513fe0d4bed67c8679eb51bca30d53
2017-12-23 18:34:49 +00:00
drh 755b0fd358 Move the generation of output column names earlier in the case of a
CREATE TABLE AS.  This is a fix for ticket [3b4450072511e62] and a
continuation of check-in [ade7ddf1998190b2b63] that fixes cases of
ticket [de3403bf5ae5f72ed6] that were missed previously.

FossilOrigin-Name: 6b2ff26c25bb9da344add79c93fb3e49fa034a89b38ef56e08e18d21de61f707
2017-12-23 12:33:40 +00:00
dan 43efc18669 Experimentally add the SQLite expert functionality to the shell tool.
FossilOrigin-Name: 51068dbaeaef13bb80af8126b8c4f3a454dee63de5127d706db50bf789533e60
2017-12-19 17:42:13 +00:00
drh 2fc865c115 Add an experimental location(X) SQL function that attempt to return the
location of the payload within the database for the record that contains
column X.  location(X) returns NULL if X is not an ordinary table column or
if SQLite cannot figure out the location because it is using a covering index.

FossilOrigin-Name: 51be9558164301c5dd4df23ab8b3e67de0b522f8d36f79f3d84d45d3dc2a83a4
2017-12-16 20:20:37 +00:00
dan 11546779b7 Merge latest trunk changes into this branch.
FossilOrigin-Name: d5b597b52a1213cdf382d96f4df3535727be0852b25bafd12bbef54da946c5f2
2017-12-16 19:16:24 +00:00
dan 2c44e370f1 Enhance the "swarmvtab" extension. See header comments in ext/misc/unionvtab.c
for details.

FossilOrigin-Name: 01c173651ab22b7b0c139eded6f2ad8504efd09088df8ae6a3471230ebf2306f
2017-12-15 20:21:17 +00:00
dan ac15e2d7cc Have the writefile() function optionally set the modification-time of the
files it writes or creates. And many small fixes to the new code on this
branch.

FossilOrigin-Name: 7b51269caebe1492885fe9b965892f49a3f8bdb1d666b0203d594c30f9e83938
2017-12-14 19:15:07 +00:00
dan 7d935ab46d Merge latest trunk changes into this branch.
FossilOrigin-Name: 803156cba8b056a1cb8d1bb186a57454afe72341abe7de1dfe529234c3415cd2
2017-12-14 13:55:01 +00:00
drh 472e41ea16 Correctly invalidate a column cache line when it is overwritten with NULL.
Fix for ticket [123c9ba32130a6c9d432].

FossilOrigin-Name: 65754dc3b830ef9482655d2e93c2cd4acaa7a8715a58d8309955d04378541f89
2017-12-13 18:01:52 +00:00
dan ece4b0c172 Add tests and fixes for the shell ".ar" command -f option.
FossilOrigin-Name: 1a9867973c9d6675fa5254fdd74f36004707a98a91593a188033cf5a49cc7a0b
2017-12-12 20:28:36 +00:00
dan d4b56e5901 Add support for parsing options in non-traditional tar form to the ".ar"
command. Have writefile() attempt to create any missing path components. And
not to throw an exception if it is called to create a directory that already
exists.

FossilOrigin-Name: 38dbeb1e777aa7ec742aa27002ad4dcee28af520dc43de96e5c56c39f16574ff
2017-12-12 20:04:59 +00:00
dan 1ad3f61b81 Enhance virtual table "fsdir" in ext/misc/fileio.c. Add support for "-C" to
the shell command's ".ar c" command.

FossilOrigin-Name: 0394889afed2479773af594e2d9659cf58b8959004ebcdeaff8e08e5dae684ef
2017-12-11 20:22:02 +00:00
dan 2ad0949db4 Add support for -C to ".ar x".
FossilOrigin-Name: 8cd70960c5ddf0d0b2c40b8b6af4ce6b0277ffdaf04f33fcb33227d2b99ad515
2017-12-09 18:28:22 +00:00
dan 88be020916 Improve parsing of ".ar" commands. Add new test file for the same.
FossilOrigin-Name: 840401cc8ce3a09e0663b46973ecd2856d9607be71d2d1e9b21f7df7a82dcbe5
2017-12-09 17:58:02 +00:00
dan 21540ae479 Add compile time switch SQLITE_ENABLE_ICU_COLLATIONS. For enabling ICU
collations without also enabling the tokenizer, the LIKE operator, the
REGEXP operator, or the unicode aware upper()/lower() scalar functions.

FossilOrigin-Name: a079f914522d7bc4b3d27d70114eb09adedfac936a64883e6ed8d382c428dd0e
2017-12-08 16:23:38 +00:00
dan f9679318f2 Fix the ".lint fkey-indexes" shell command so that it works with WITHOUT ROWID
tables.

FossilOrigin-Name: 5771b1d611b3562ea6c040f3f893073c4d0ee58c35b37ae211520d9aee8ed547
2017-12-01 18:40:18 +00:00
dan 92b67025d4 Fix some minor problems in test scripts.
FossilOrigin-Name: 4c551fdebc7feda3dcfeec719387d879cd5e2cbe213c0c1aac0a965b3f9e882d
2017-11-30 11:21:59 +00:00
dan 6302590677 Update test file walprotocol.test to account for the changes in the wal
locking protocol.

FossilOrigin-Name: 64e567009dd56ef595850fe460925bc15fa875163541527638b654aa2b2cf785
2017-11-30 07:55:15 +00:00
dan 3d9c7c3a4f Merge latest trunk changes into this branch.
FossilOrigin-Name: 373fa21bffd37053bb1a741d3ace2f003a5639bf373c20ea7bb24a31080888b3
2017-11-29 16:16:29 +00:00
dan 0895117512 Fix an assertion fault found by OSSFuzz.
FossilOrigin-Name: 75d699877fa7d06d30285ecf008fbedfdf68cc7965bb328c96f5a931d1f13f04
2017-11-28 20:43:40 +00:00
dan 05573e41a8 Lock the wal file for all snapshot transactions, even if they would not
otherwise require this, preventing checkpointers and writers from wrapping the
wal file. This means that if one connection has an open snapshot transaction
it is guaranteed that a second connection can open a transaction on the same
snapshot.

FossilOrigin-Name: b81a31495bd27c1d96f7df653da3502054240cb5acf66b860da7f0f9b422a524
2017-11-28 13:39:41 +00:00
drh cb45eef4d1 Update tests to deal with SQLITE_FAST_SECURE_DELETE.
FossilOrigin-Name: e6b89304695be371978e65dddd710c8bd563c66b9c94d23165142b6c235c82e1
2017-11-28 00:52:14 +00:00
dan 7ac2d48eba Fix a faulty NEVER assert() that could fail for SQLITE_ENABLE_STAT4 builds
that use foreign keys.

FossilOrigin-Name: 465350e55ddaf30cfba7874653301de7238a9dc2bc5e1f800fc95de9360679f6
2017-11-27 17:56:14 +00:00
drh 53bf7175af Fix a problem in the omit-table-from-left-join optimization
from check-in [0cd82ee9a8413cf] that was discovered by OSSFuzz.

FossilOrigin-Name: b016c28fa5617a20ad34c005372e738d28f7fc4388d19ee0cb7add4ed19d74aa
2017-11-23 04:45:35 +00:00
drh fa337cc115 Completely disable the skip-ahead-distinct optimization for all but the
inner-most loop of a DISTINCT join.  Fix for ticket [ef9318757b152e3a26e9592]

FossilOrigin-Name: b7595cf2cadcba486e60b3c230ccc412a7cf449c1d56cbe65869e6d5d9f1374d
2017-11-23 00:45:21 +00:00
drh 065b34f6e5 Fix the skip-ahead-distinct optimization on joins for cases there the table
in the inner loop of the join does not contribute any columns to the
result set.  Proposed fix for ticket [ef9318757b152e3a2]

FossilOrigin-Name: 2dcef5a9ae7f347da65207bf6bf612fb12e18e1a6704799322f0cf2a86154cfd
2017-11-21 23:38:48 +00:00
dan 41203c6cf1 Update the omit-table-from-left-join optimization so that it can omit tables
from the middle of the join as well as the end.

FossilOrigin-Name: 618ca9fe53d8d2d7b4f368e6ee404d5fceeecac0d689f32ab62af8a6cbb37401
2017-11-21 19:22:45 +00:00
dan 17f188e3cf Fix a typo in a test script on this branch.
FossilOrigin-Name: bff5dcfd2b29ee4834258914410a5dee69ec2727dd254053e3ebaf5090937694
2017-11-20 15:45:03 +00:00
dan 75dbf68b5b Fix a problem preventing the planner from identifying scans that visit at most
one row in cases where that property is guaranteed by a unique, not-null,
non-IPK column that is the leftmost in its table.

FossilOrigin-Name: 299d7ca52fec32f04ffd2b8561dd4b839e891792162f8b00259368683436b02d
2017-11-20 14:40:03 +00:00
dan 6bcabfe1a3 Add some missing "finish_test" lines to the end of test scripts.
FossilOrigin-Name: c21406ab3281480d3eddca0cdf5aea3abc224425ee52c10eed3ff702a0ae5c26
2017-11-17 20:07:19 +00:00
dan 6cbc5074e8 Avoid creating a master journal if all or all but one of the databases in the
transaction is a temp file.

FossilOrigin-Name: 355d1232fd7314723afaa8d6b8b73506b09cbba1113a88b10204ba89be993508
2017-11-17 08:20:10 +00:00
drh 3dbb4acf07 Add a missing "finish_test" to the end of the stmtvtab1.test script.
FossilOrigin-Name: e0b5c0585e8530bd516b340093a46d8e358a731facb78da22d3208633c852804
2017-11-16 19:04:33 +00:00
drh 65efeaca83 Add the ability to read from read-only WAL-mode database files as long as
the -wal and -shm files are present on disk.

FossilOrigin-Name: 00ec95fcd02bb415dabd7f25fee24856d45d6916c18b2728e97e9bb9b8322ba3
2017-11-14 19:34:22 +00:00
dan dc32b448d3 Add further tests for the code on this branch.
FossilOrigin-Name: f8c4e33f4813e0c909064406b5cc17e2d465d8a48a50ede1d356b39479d3d669
2017-11-10 17:47:54 +00:00
dan ca3e3c3f1a Fix a problem involving "DELETE/UPDATE...LIMIT" statements that use an INDEXED
BY clause.

FossilOrigin-Name: 09f94c2c8199b0d23a45cc062ca9561f9e5ddfcba117100e41889ce199d21bdb
2017-11-10 16:14:26 +00:00
dan 26caf5bef1 Fix a problem with (DELETE...LIMIT) statements against WITHOUT ROWID tables
with a single column PK.

FossilOrigin-Name: 35477a3dcceadf5dade8e036d5a2ce91b9ca83c4b85d309db233bdbcf538b1cc
2017-11-10 15:42:21 +00:00
dan e941877806 Enhance walro2.test to better ensure that readonly_shm clients are not using
invalid *-shm files.

FossilOrigin-Name: ff630b66714b20c09888ead0a45f344d63e0d9a5208867d6266e74f79187076c
2017-11-09 20:34:35 +00:00
mistachkin bcb416a9ff Get read-only SHM file tests passing on Win32.
FossilOrigin-Name: abef05353554e72f4d08aff562b87ff8530e8537a79e58d831205ea8c46eed07
2017-11-09 20:02:44 +00:00
dan b3c16b899b Add SQLITE_ENABLE_UPDATE_DELETE_LIMIT for views and WITHOUT ROWID tables.
FossilOrigin-Name: 584b88aaf89ab30cb51185396b7b50c6ecba286add023a26ab41d865b9c605ce
2017-11-09 19:53:06 +00:00
mistachkin 4ff8431fd1 Initial work on porting the changes on this branch to Win32.
FossilOrigin-Name: 3738bfd0c0eadb10eea58954af5052cb6ce164059f3aacfe65d7da6a400c63c7
2017-11-09 16:30:55 +00:00
drh 2fba394c97 Disallow ORDER BY and LIMIT on UPDATE and DELETE of views and WITHOUT ROWID
tables.  This is a temporary fix for ticket [d4beea1633f1b88f] until a better
solution can be found.

FossilOrigin-Name: 62fe56b59270d9d7372b1bb8a53788a40d20d0f111fe38c61dd6269848592c70
2017-11-09 03:55:09 +00:00
dan 08ecefc5b1 Handle the race condition that may occur if another process connects and then
checkpoints and truncates the wal file while a readonly-shm client is building
its heap-memory wal-index.

FossilOrigin-Name: 5a6703fc3f2174b3e9a624c7272ae013b73c42d6c97ffa62b58553efdb54e3bc
2017-11-07 21:15:07 +00:00
dan 85e1f46eb9 Fix a problem causing an INDEXED BY specifying an unusable partial index to be
mishandled.

FossilOrigin-Name: 292a04086a902634fc514b379a2b245eb2681c1b84d9bb950b6ecb9aab28b468
2017-11-07 18:20:15 +00:00
dan f12ba66cf8 On unix, if the *-shm file cannot be opened for read/write access, open it
read-only and proceed as if the readonly_shm=1 URI option were specified.

FossilOrigin-Name: ba718754fa5ab8596cb84b751051de98afa2706fe6c5df39ad6d925d790719ee
2017-11-07 15:43:52 +00:00
dan d24c5b1c98 Add fault-injection tests for the code on this branch.
FossilOrigin-Name: a7d949fb735f60c19e7257a1a7a12568a9c15be9cd980c018f3a0d6bf112c339
2017-11-07 09:08:43 +00:00
dan ab54838452 Add further test cases for the new code on this branch. And a couple of fixes.
FossilOrigin-Name: 71af9acb227a91d9ad8798c9d0b12d6967e863d050f5cb1fddb45f25ee1f47db
2017-11-06 19:49:34 +00:00
dan cbd3321978 Add further tests for the code added on this branch.
FossilOrigin-Name: a6716fcde38b28b8a03b40f9d16f78a57ec20f60cf391ff553692641cb7f0d3f
2017-11-04 21:06:35 +00:00
dan 11caf4f4b7 In cases where a readonly_shm client cannot take the DMS lock on the *-shm
file, have it parse the wal file and create a wal-index to access it in heap
memory.

FossilOrigin-Name: 18b268433d739486eac1b04947bd418655e4bc56e8dc63ffa558aa4552a32e30
2017-11-04 18:10:03 +00:00
dan 44c8a97e01 Fix test cases in wal2.test broken by the locking change in the previous
commit.

FossilOrigin-Name: f569c3517234881f9425075aab65a32ffd0deb8e793f421a241d8cca881da33f
2017-11-02 18:57:46 +00:00
dan dea5ce36f5 Avoid locking shm-lock WAL_READ_LOCK(0) during recovery. Doing this allows
recovery to proceed while a readonly_shm connection in unlocked mode has an
ongoing read transaction.

FossilOrigin-Name: 5190d84a296b7cf716ef43bf7b6d4d351ef1a4d650de37dc01a5ab333da7c05d
2017-11-02 11:12:03 +00:00
dan 92c02da33e If a readonly_shm connection cannot map the *-shm file because no other
process is holding the DMS lock, have it read from the database file only,
ignoring any content in the wal file.

FossilOrigin-Name: ce5d13c2de69b73378637d4f7e109714f7cd17bf1d1ad995e0be442d517ed1b3
2017-11-01 20:59:28 +00:00
drh 37ab9523fa Move the test scripts for checkfreelist and checkindex over into the
ext/repair/test directory.  Run them now using the sqlite3_checker utility
with the --test option.  Some tests are currently failing due to an
incomplete port.  This is an incremental check-in.

FossilOrigin-Name: 17f8d5e111a9fe5b074f946e23936ae5a2a7a8d8018bad4212660d8eb81c04b7
2017-11-01 18:05:32 +00:00
dan 514c4b7e5d Merge latest trunk changes into this branch.
FossilOrigin-Name: 985bfc992950625a45a7521bf4c8438cd0170de974dff976968be158ac5922a9
2017-11-01 07:06:41 +00:00
drh 1884d00dd0 Merge all recent enhancements from trunk.
FossilOrigin-Name: 24adf90ffb3ce9ff3c26efef0357f3a47312e8d11dc391ef2cc7e6873ef25895
2017-10-31 18:15:36 +00:00
dan 7eded5925b Fix a couple of issues in incremental_index_check to do with CREATE INDEX
statements that contain embedded SQL comments.

FossilOrigin-Name: 2aef41815a9f1786ebdf09d8f6cfa59a8e7d733253eafeae24fa6e2a093bb1d8
2017-10-31 12:01:01 +00:00
dan e7a579d27d Add support for indexes on expressions to incremental_index_check.
FossilOrigin-Name: 8c1c701fdbe0d56ee7f6f7d7b583aafde9fa14acc93ee8ecaddc8bb311e2bf52
2017-10-30 19:38:41 +00:00
dan 75650d4f94 In checkindex.c, use C code instead of SQL/group_concat() to compose various
SQL clauses. This is to make it easier to support indexes on expressions.

FossilOrigin-Name: 940606b3af059eb3f79d71fec871ea88df8bce0349f5b33b79c147a85610e269
2017-10-30 17:05:18 +00:00
dan 6fc6617909 Fix an issue in incremental_index_check with indexes that use non-default
collation sequences.

FossilOrigin-Name: 3ebb2351e2650d263029d2c0042683cba3529c9d3f76b5f994f2e737b84d3f67
2017-10-30 08:04:38 +00:00
dan 19465e7380 Fix "after" parameter handling in the incremental_index_check code.
FossilOrigin-Name: c40c3c62e996044f31ca49ffc2edb2cc0320e69956f7ee6fe3e9012200e0d9a0
2017-10-28 20:31:25 +00:00
drh 6d6e76f75f Add test cases from OSSFuzz to prevent a regression in co-routine
processing.

FossilOrigin-Name: 689743d8e3fa81e65dcb067bbf61bab09210b3b39586c865c00d9f1d6692daf2
2017-10-28 12:20:09 +00:00
dan bde13e2694 Add the start of the "incremental_index_check" virtual table in
ext/repair/checkindex.c. For incremental verification of index contents.

FossilOrigin-Name: d5b9dada471358a2864727759648b763bf6890fc2521fac53c0d8216017d39b7
2017-10-27 20:53:16 +00:00
drh 0503f2acfa Fix a Pager ref-count leak in the sqlite_dbpage virtual table.
FossilOrigin-Name: c063bb04da77d6847e4e254b6bb939c860e781382e34ddff8e0454d9db7552e2
2017-10-27 18:24:11 +00:00
drh 6e61c5f895 Additional test case demonstrating the ability to access structure elements
using quoted names in the JSON1 extension.

FossilOrigin-Name: 8d3cc928a8f0c7b2616c5c75af9d3a50bd4f0fe8e4ccab545ab8648cbfbb4b7f
2017-10-27 12:27:25 +00:00
drh 47d9f83982 Enhance the min/max optimization so that it works with indexes on
expressions.

FossilOrigin-Name: 6caec9082b3ad6d3f89f6291084a8f5c80f296630e2e94bc764389ac1a47a833
2017-10-26 20:04:28 +00:00
dan ab04eff809 Fix an error in the previous commit on this branch.
FossilOrigin-Name: f71dfee06ce1e0eee760cfca19482bdec7729d6c7d28f10f4cfd21e1f92a04b0
2017-10-26 17:34:50 +00:00
drh 3cd8aaa748 Fix the sqlite3_dbpage virtual table so that it can read and write from
any attached database.

FossilOrigin-Name: d4f893e1ae53a0445939ea2920af87d21dd36270494381028b2eaebe5c188f18
2017-10-25 19:18:33 +00:00
dan 685bea66ce Merge trunk fixes to swarmvtabfault.test and the lsm extension into this
branch. No changes to core SQLite code.

FossilOrigin-Name: f632b87002a7ee9be569aacb5ce85bb7e9c86b7d77bb8d1bf8b55eb5be4ecc77
2017-10-23 20:20:42 +00:00
mistachkin 0295774ca0 In the 'swarmvtabfault' test module, make sure to close the database handle prior to raising an error from the 'not found' callback script.
FossilOrigin-Name: f317037b31b19007e30bb41c54506d9d241a6d590c2b31e2ddda10d4f6e7605b
2017-10-23 20:17:19 +00:00
mistachkin 15dd780a9e Cleanup a test database file prior to using it.
FossilOrigin-Name: e1bf5e1d06dbe573671c6b5472b45f5bfa3cf82374f70a76b4c80815bf15a29b
2017-10-23 17:36:52 +00:00