Commit Graph

25154 Commits

Author SHA1 Message Date
drh
5b046dadfc Fix harmless compiler warnings.
FossilOrigin-Name: 8d5b76593d82b3a57bc904ced33c24cd49369868b5dd2fe7f2c7b114f5aee2f6
2022-11-28 20:08:15 +00:00
drh
57ba01de4a Implementation of enhancement request [695a1a53dea6b240]: The query planner
has improved awareness of when indexes on expressions are covering and adjusts
their costs accordingly.

FossilOrigin-Name: e3474d79b27298e96e7686e5a6f9a8e736b8a6568a0a08fb1abe5bcca038ca63
2022-11-28 19:42:48 +00:00
dan
1b3d13e65e Add the SQLITE_FCNTL_RESET_CACHE verb. Use it to ensure that the page cache is purged before and after a the recovery extension is run.
FossilOrigin-Name: 6db0bc4bc0d272b610bef2aeeae43f539ed6e7cc0a9cc767d5af85ecb0019d5f
2022-11-28 18:41:41 +00:00
drh
cbb9585805 Test cases added derived from the enhancment ticket.
FossilOrigin-Name: c022c0152ad61a4f56e36f4062609073e2273fbf6f826c20652159be229c2d46
2022-11-28 18:17:42 +00:00
drh
8ffa77f3a9 Merge trunk changes into covering-index-expr branch.
FossilOrigin-Name: 89d775ada39aed4dc532374ace02156d07dc06e8ae54a194608af0c5a582d20f
2022-11-28 16:11:10 +00:00
drh
84606c9737 Rework the covering index checking routine, whereIsCoveringIndex(), so that
it can return a "maybe" result for aggregate queries where we are not exactly
sure.  The index is scored as if it is covering, but the main table is
still opened.

FossilOrigin-Name: b8eec4214363192e6f3e12b3faa5810d8269a5fdaecab3ec09b02e5002cf798a
2022-11-28 15:23:53 +00:00
larrybr
e862b5fe1d Add (optional) base64 and base85 UDF extensions.
FossilOrigin-Name: b44ab10c49bc2895483a9d40813be3798710ee713cc4bf04e449dce55a68452a
2022-11-28 14:51:50 +00:00
larrybr
d8d68a5b8e Sync w/trunk, zap surplus space.
FossilOrigin-Name: b8345630a2a322234bda49ee4b996f6ba20e2b080621e229a2ec5e820892a663
2022-11-28 14:11:48 +00:00
dan
abbfa7aa17 Ensure the RETURNING clause is honoured when a row of a temp table is updated by an ON CONFLICT clause.
FossilOrigin-Name: a2449bcc2c71d0f4c3289621fbf1cb97f0f407c9f7b5bf18245b7854a07c6cfa
2022-11-28 13:47:27 +00:00
larrybr
c1ca183abf Conform CLI .trace arg handling to its help.
FossilOrigin-Name: 31546ea320e8daa020f8a9f491718ffc3dde0e32954018afed88e8494a6aff6c
2022-11-28 02:28:44 +00:00
stephan
ca9309cc56 Document an OPFS API change in Chrome v108 which does not break our code but does change several formerly async methods to synchronous. No code changes.
FossilOrigin-Name: e9dd87e28a96eb6560943b9a2cb3cb0ff14698c96fe63865944dcfed73b67bd3
2022-11-27 00:57:56 +00:00
drh
9d34152bcf Better reporting of when the WHERE clause analysis thinks that an index
is covering.

FossilOrigin-Name: 17ebcf316b91042c823eea7bb6f1309325023cb5c70538cdb2ce932caee2ef05
2022-11-26 20:52:38 +00:00
drh
217e77d240 Show which AggInfo.aCol an Expr node represents in the tree trace debugging
logic.  Does not affect production builds.

FossilOrigin-Name: b8076f4dc55810855f668dfd4221bdb555cc78c61df200cf0cb87eb137e22850
2022-11-26 20:48:56 +00:00
drh
8beb7f56f5 Merge trunk fixes into the covering-indexed-expr branch.
FossilOrigin-Name: 9ac67ff968e874cf955e46e3993e3215c766feec3d5bdd38d77884eedd86b59e
2022-11-26 20:13:54 +00:00
drh
743c9ec4a5 Fix an incorrect assertion-fault in the "TREETRACE" debugging logic on
select.c.  The problem does not affect production builds.

FossilOrigin-Name: f11bb42292d94e7b7687bd2134f72afe1353182238cb4837fc0a6d78a021dd7e
2022-11-26 20:12:02 +00:00
stephan
4245667e35 OPFS VFS: add the opfs-unlock-asap=1 URI flag which tells the VFS to release implicit locks ASAP instead of during VFS idle time. This improves concurrency notably in the test app but brings a significant performance penalty in speedtest1 (roughly 4x slowdown). This is not the final word in OPFS concurrency, but gets us a step further.
FossilOrigin-Name: 9542f9ce9e023b489e2d93661f719fb0751c1e28f72fded9d3c2156d5777e7b1
2022-11-26 15:24:58 +00:00
drh
6e71186904 Experimental changes to help the query planner detect when an expression
index is coverting.  Works somewhat, but there are tests that fail.

FossilOrigin-Name: 968c189bcf29a9b25305251a58d09b7d52ab9dd08f5057dc3ab1f7ad1a5316a0
2022-11-26 14:19:47 +00:00
drh
902ea83925 Relax restriction (8) on the push-down optimization so that it only applies
if one or more columns uses a collating sequence other than BINARY.
See [forum:/forumpost/3824ced748baa808|forum post 3824ced748baa808] and
check-in [346a3b12b861ce7b].

FossilOrigin-Name: adbca3448e2099f0d6149a073978f230ed9a92a2f384779879ef89e672231bcf
2022-11-25 17:05:55 +00:00
drh
49d400b091 Fix #ifdefs so that restrictions (8) and (9) of the push-down optimization
are still enforced even if compiled with SQLITE_OMIT_WINDOWFUNC.  This
fixes a bug introduced by check-in [346a3b12b861ce7b].

FossilOrigin-Name: 09e1e42e0ff26f9a71cbd128169f060a66425828d637bf8f781490ca38d99103
2022-11-25 16:32:59 +00:00
drh
ae5cfda656 Immprove the query planner such that it is able to make use of indexed
expressions within an aggregate query with GROUP BY.  This implements
enhancement request [99378177930f87bd].

FossilOrigin-Name: b9190d3da70c41717eb188474fd225ee43d0b46646e1b03de5967bd332553870
2022-11-25 16:10:48 +00:00
drh
23f61a4ba8 Add restriction (9) to the push-down optimization: If the subquery is
a compound then all arms of the compound must have the same affinity.
dbsqlfuzz 3a548de406a50e896c1bf7142692d35d339d697f.

FossilOrigin-Name: 1ad41840c5e0fa702ba2c0df77a3ea126bd695b910b5d1271fa3129c38c58f5f
2022-11-25 15:52:00 +00:00
drh
bedbb1bfab Remove a redundant assert() statement.
FossilOrigin-Name: e3e1b453dc47884ddc9e51c4302fa2f4d40bca3d69ac7c13d8e2ae1adc81ac56
2022-11-25 13:26:51 +00:00
drh
f5d0656f16 Improve the robustness of the updateAccumulator() routine against OOM.
FossilOrigin-Name: d2844f577b69fdc16a0a2568c0958fc3d8aff33e9a0ef80e0f58e92f01097432
2022-11-25 13:15:48 +00:00
drh
7960da0346 Improved comments. Add assert()s to verify that the AggInfo structure
is unchanged after registers have been assigned.

FossilOrigin-Name: 5200b84195ee1ccaa387f7032eae3d463724c48cb53ba0251bbc79e927dd9752
2022-11-25 13:08:20 +00:00
drh
8683c09281 Correctly deal with IF-NULL-ROW operators when dealing with
indexed expressions in aggregates.

FossilOrigin-Name: 939cb47025354e2df047de7654c0b06f791957cfe4e904abe8892207cea90215
2022-11-24 23:35:27 +00:00
larrybr
472de149ae Speed up base85() conversions and sync w/trunk.
FossilOrigin-Name: 17b823500a2ed135c1f40aa7f4d87ba5b2eab35c0abd9e0856041cf0f510cbee
2022-11-24 20:11:34 +00:00
drh
e644f7c24b Take care not to try to add comments to a TK_AGG_COLUMN opcode that does
not have an associated Table object because it is a reference to an indexed
expression.

FossilOrigin-Name: 5fc23863e4a6388884ce061cd69546757b4c39d731bbb58a8121b92fd5144f7b
2022-11-24 19:02:40 +00:00
drh
594357f2e1 Change a NEVER() into an assert().
FossilOrigin-Name: a0fd44f4e5a0ec83465203a0009f307ca50223833575895a3ebf8a289515714f
2022-11-24 18:45:56 +00:00
dan
eb84c81c53 Fix a test case in fts3expr4.test to account for different locales.
FossilOrigin-Name: a2b6883ac2ef878f525ee847b170beb9f9ab9d1fa67557101be2cdae1e7f7a57
2022-11-24 17:58:55 +00:00
stephan
df5d06d03e More work on the OPFS concurrency testing app.
FossilOrigin-Name: c0458caca3508d5d252f9b5198bda4f51a5c1874540f014b17e409f2daab1706
2022-11-24 17:53:09 +00:00
stephan
6b86b33793 JS documentation cleanups. No code changes.
FossilOrigin-Name: 18e89a436daa18a8c972caf06b298da43c97a6ea3e2e5229dccb6920c27bfdb9
2022-11-24 16:03:49 +00:00
dan
647b0dd12d Update multiplex3.test to account for the fact that the multiplexor xDelete method may return an error even if it manages to delete the first chunk of a file.
FossilOrigin-Name: 1a7f3254735054ed8ca32d5ec7c8cde9195a64702638bdc50392007e396fead2
2022-11-24 15:32:00 +00:00
drh
c25f5ea6e8 Add NEVER() and ALWAYS() macros on branches that are believed to be
unreachable.

FossilOrigin-Name: 3a901e88c87fc76c7fe42e47a976a5706830f0dbd6027605663e4d55f4f33590
2022-11-24 15:04:23 +00:00
drh
b669bb5e2b New test cases. Fix the logic so that it works for GROUP BY aggregates
that do not require sorting.

FossilOrigin-Name: ef6ebe7922f56c1584a005deedc85ca1070b4fe5082ada8bbf8d06df54f1c9ef
2022-11-24 13:19:25 +00:00
larrybr
9b9017def4 Speed up base64 conversions, and add test with more data for the baseNN conversion to grind.
FossilOrigin-Name: 6c84ae4ba83713c751fddff8be41686bbcb525ac8135e1520436c62d0bc23d2c
2022-11-24 02:59:33 +00:00
stephan
f46091d73f Add sqlite3.oo1.DB.prototype.checkRc() and tests for both that method and its class-level counterpart.
FossilOrigin-Name: f7eaa6ba2147bfd6dbdc2444d0f919d846aa7f9b68cccab17ef585ffdacf3d60
2022-11-24 02:35:03 +00:00
drh
4b1555a9fc Merge the latest trunk changes into the agg-with-indexed-expr branch to
simplify diffs.

FossilOrigin-Name: 38c3d3f1ed0fd2bb62aa8a7e5a27f2b247123e094e2fdb0a2475d788c3dfbc04
2022-11-24 01:41:44 +00:00
drh
9b80cb5f39 Add explanatory comment to the new optimization. And add a test case.
FossilOrigin-Name: e6c20f61de7d048eee65c8e74a3eb36760ab9747ebd1ab50e49642b777c10306
2022-11-24 01:40:20 +00:00
stephan
f1ce4f40c7 Minor JS doc updates.
FossilOrigin-Name: 27efd63ad7fb3bf8d0d07f2c9f48652c8cacc4e697c229c8085120a8e6b50a39
2022-11-23 21:09:51 +00:00
stephan
875db41afc Add optional zSchema argument to sqlite3_js_db_export().
FossilOrigin-Name: 9c23644b1e5bf44bfb431a35fd1674c11ccb99e9eb0989f10175b0cb2a858eaa
2022-11-23 21:03:22 +00:00
stephan
056a71562f OPFS concurrency test: add a URL flag to enable/disable unlock-asap mode.
FossilOrigin-Name: 1c1bf22eadae2a5a7d4358e7cdd22641c2efb9296f42e7376749293b3a58b114
2022-11-23 20:49:08 +00:00
stephan
e79cb67c35 Add an experimental OPFS VFS-specific URI flag, opfs-unlock-asap, which tells the VFS to release implicit locks ASAP. This permits higher concurrency but hurts performance considerably. This may or may not be obsoleted by other concurrency-related experimentation.
FossilOrigin-Name: d23c917013485ec2793125221f3936b05c39d6eca941629fb819b6b4aa714520
2022-11-23 19:03:22 +00:00
drh
2e30d95fb6 Aggregates with GROUP BY now make use of expressions on indexes. This code
works and gets the correct answer for the test case in the ticket.  Lots more
testing and documentation is needed, however.

FossilOrigin-Name: 8dcf9f2031c16f296d187fe876d4204c71fc96fec120984ff11b6d8b03d58a5f
2022-11-23 18:51:04 +00:00
drh
590f013a87 This attempt at modifying AggInfo to make use of indexed expressions does not
work.  It gets an incorrect answer for the test case shown in the ticket.

FossilOrigin-Name: 84c06023f4a1606664fdb9811312603b31f7c94a43d0e443ba7dde7fdba029e3
2022-11-23 17:56:00 +00:00
stephan
ad1285c5c0 Initial infrastructure for adding a mode to the OPFS VFS which causes implicit locks to be released ASAP, which increases concurrency at the cost of performance.
FossilOrigin-Name: c5b7a9715a13b696ab3ee965aa0a310f59b65f07cecd72faa2e3504bfd8eb632
2022-11-23 16:39:07 +00:00
dan
c32e16643d Update Makefile.in to include new target "sqlite3r.c". For generating "sqlite3r.c" and "sqlite3r.h", versions of the amalgamation that include the recover extension. To build the shell tool against these files, add -DSQLITE_HAVE_SQLITE3R.
FossilOrigin-Name: 5f135575b923cb59947667071c6af9ff14063c933cedf37d6c2a0a1b86c85032
2022-11-23 16:08:49 +00:00
stephan
91a81316b9 Remove a bit of over-cleverness which breaks loading of sqlite3.js in some main-thread cases.
FossilOrigin-Name: 220cc4c6399b772b4ece03305a41b437ef0654d586a8a1c3dc5e7606fd36d655
2022-11-23 15:52:52 +00:00
stephan
3eaaec945c OPFS concurrency tester: ensure that the work interval timer does not overlap with the work time.
FossilOrigin-Name: 8f2076553bc486ea6a17934695ecc02217461af2082d891697b62aab89bd1b43
2022-11-23 15:33:49 +00:00
drh
c6138e970e Further foundation prep work prior to starting to flesh-out the
optimizeAggregateUseOfIndexedExpr() routine.

FossilOrigin-Name: 23145fe999ff74d787a3999baedd4ffe755c5f1f1275082ed0338ba637ecc56e
2022-11-23 14:13:39 +00:00
larrybr
809be56218 Add more baseNN tests, get oversize error trapping working, and sync w/trunk
FossilOrigin-Name: 03819e9368fd9f78f351147a1dc865743f9634893e43a9d1e3d7cbaf4c966069
2022-11-22 22:46:41 +00:00