mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,437 Commits 1,487 Branches 166 Tags 416 MiB
57645b67fa
Commit Graph

26437 Commits

Author SHA1 Message Date
drh
7d70571142 Merge the latest trunk enhancements into the json-opt branch. 
FossilOrigin-Name: 00bfc4918be49ac74a3e7851c88ab7ec226e6a37853f8ad4c77f758751960456
 2023-07-24 12:37:23 +00:00
stephan
479cfd5af3 When writing the filename header in a opfs-sahpool file, ensure that all remaining bytes in that part of the header are zeroed out to avoid the downstream problems reported in [forum:d50ec48a293988a5|forum post d50ec48a293988a5]. 
FossilOrigin-Name: c1b080e39397c983c13a5e79303223827de7b4946c18a79396851ec1814782f3
 2023-07-23 22:14:41 +00:00
stephan
2ecadd8869 Add the opfs-sahpool sqlite3_vfs implementation to JS, offering an alternative to the other OPFS VFS (with tradeoffs). 
FossilOrigin-Name: d2e602cda44bf35e76167143262b4f91826d25780d0e095e680a31d5dedb2018
 2023-07-22 19:57:42 +00:00
dan
2159292ce0 Integrate contentless delete with auto-merge. 
FossilOrigin-Name: 85c1589ab1fc69d1eef4bbc1bdefa2b10af5f6b9c08e813130b93829b592f416
 2023-07-22 19:47:46 +00:00
drh
5d03b1610f Do not read past the end of a text buffer looking for a zero terminator, as 
that space might not be initialized.  If the buffer is owned, just set the
null terminator.  This is a better fix for the OSSFuzz-detected
use-of-initialized-value problem.

FossilOrigin-Name: 931bccb0cc290b8bf3027641e7a7fac30e3244d7dc84aa9e38b24b7e9544ca06
 2023-07-22 16:37:28 +00:00
drh
61a5b6e3bc Fix a harmless use-of-initialized-value warning from OSSFuzz that results 
from recent large string optmizations.

FossilOrigin-Name: 1a6b3dd1c40277a0d0f0bb562ddc4868aadd632fc2d29be1b17bb33fc22c46c8
 2023-07-22 15:21:41 +00:00
drh
b2ee272075 Performance enhancements for queries using very large string values. 
FossilOrigin-Name: 3661b9cd98565ea23b0cafd030668f09885dcbd54eb12b7b3cd85c289aef4c3c
 2023-07-21 22:28:34 +00:00
drh
21f22f9088 Mark an unreachable branch as NEVER(). 
FossilOrigin-Name: 414a4d660c3b505640227066b468db1268fcb1bd3fad931644a769617c6c00cd
 2023-07-21 22:22:13 +00:00
dan
330e36c2c6 Fix fts5 incremental optimization so that it too can handle an index that consists of a single segment with one or more tombstone hash pages. 
FossilOrigin-Name: e61c9b083f5e0b6b6ee18f9394581ad816f445dbfb72ed1fe954f4182755a576
 2023-07-21 21:10:33 +00:00
dan
4e0c157d02 Ensure the fts5 'optimize' command correctly rewrites any index that consists of a single segment and one or more tombstone hash pages. 
FossilOrigin-Name: f4926006b371d9a1439a25384bd50a50c2f1c03f75a7c2c3134ae72abb971c91
 2023-07-21 19:33:35 +00:00
drh
89f07b1276 Fix harmless compiler warning introduced by the previous check-in. 
FossilOrigin-Name: 06f829e9e57c5aa495f519ad7bb379be611e8e21ef60d36e63d055e45f7a6117
 2023-07-21 19:17:26 +00:00
drh
028acd974f Avoid a potentially large strlen() in sqlite3DecOrHexToI64(). 
FossilOrigin-Name: 5413b02bb629b9cbb76f7e688e94ebcf24276c01436d3feb73ff57c036e1d2aa
 2023-07-21 18:38:59 +00:00
drh
569700a72e Further improvements to large string handling in relation to JSON. 
FossilOrigin-Name: 1e5df0aa3dae5cadbf1d07c718ae2a5212543300b68e49d35e8c96855a7f619c
 2023-07-21 18:09:07 +00:00
drh
eee8687a9f Fix harmless compiler warning about signed/unsigned comparison. 
FossilOrigin-Name: 75cc3c89ee2dcfefa9421ce60bee77e85d2895918e8c5cfd05c434f8932a99b5
 2023-07-21 15:07:56 +00:00
drh
7a2280fe65 Multiple optimizations that try to preserve or infer the zero-terminated 
property of TEXT values.  Avoid unnecessary copying of text values destined
to become function parameters.  All changes help improve performance of
doing UPDATEs on large JSON values that are indexed multiple ways.

FossilOrigin-Name: d0278cdedfa04fb0b61838ab9622be8a2c462f58d5c3ebc4c5f802a727d0974e
 2023-07-21 15:01:53 +00:00
drh
c8f4fec2de More infrastructure changes towards improving JSON cache performance. 
Incremental check-in.

FossilOrigin-Name: 1955e66cfc4614df97b8d68b0e662f309513d62dc8aeec71af5a54e66b79c707
 2023-07-21 11:09:53 +00:00
stephan
bfe6dd0100 Minor internal cleanups in the opfs-sahpool VFS. 
FossilOrigin-Name: 74ad31e2908af8225b7aa527dbcd1877423d58163e365317a78453b31e322ea3
 2023-07-21 10:51:35 +00:00
stephan
3068d6622e Internal cleanups in the async part of the JS library bootstrap phase. 
FossilOrigin-Name: b6d57ab63793241a500ea527c5b3216c54b3ff1972d3adbbf42a9a53bfec0aa1
 2023-07-21 09:10:42 +00:00
stephan
35325a37ce Minor internal cleanups in JS code. 
FossilOrigin-Name: 21a2ca9fc46bf746874579897872e2a45cb07f278abb670dd22b122f7d6a9a6c
 2023-07-20 23:25:32 +00:00
dan
3a51f8c307 Fix some divide-by-zero errors that could occur when handling corrupt tombstone hash records. 
FossilOrigin-Name: 7567ca0676f0d45026f5cd4f3fbcd09119c2eaab8ec1711499609c16c452b5e4
 2023-07-20 20:29:56 +00:00
dan
d6f5aa824e Avoid an infinite loop that could be entered when dealing with corrupt fts5 tombstone hash pages. 
FossilOrigin-Name: 69ce2ce035279f2a00c2238187cf4d2a9092c3410f5900e4613fe4e46311169e
 2023-07-20 20:09:26 +00:00
drh
51d7869096 Experimental framework upon which to build a better JSON parse structure that 
supports cached of modified JSON.  All of these changes are tentative and
subject to change or removal.  Incremental check-in.

FossilOrigin-Name: e384163a4763c10b5838cbf28f9c4d8ea799bbbed6e890e8aa91b2bd1458646d
 2023-07-20 17:45:09 +00:00
dan
0b3791b537 Add tests and fixes for the new code on this branch. 
FossilOrigin-Name: 5aac50e92e956b15367c75c20c17bc1c75e84e2752bfffe4ad0a266cb9bd3b8a
 2023-07-20 16:07:04 +00:00
stephan
0b518b8d65 Filter the OPFS VFSes out of the sqlite3-node.mjs build. Add another level of subdirectory to the sahpool to later enable transparent support of client-provided files under the VFS's root dir. Rework the awkward sahpool-via-oo1 mapping. 
FossilOrigin-Name: 080a4d0aba30d8f3802b49be4a113205f069b3bdea8cebf525d654055642ff62
 2023-07-20 09:06:42 +00:00
drh
f9bfc32b73 Performance enhancements to the JSON parser and code generator. 
FossilOrigin-Name: 0a745897d66e9f36ea6e787fad54f0c413d52c226dc96fad36ccd33a0aa1930e
 2023-07-19 23:02:45 +00:00
drh
93b3c1f398 Revise the new JSON parser performance test to make the test run 25 times 
longer, and thus provide a more repeatable number.

FossilOrigin-Name: bee9e403ae47103938aabb9a7a7e120dfa741b464875965e58788a51fa56a8fe
 2023-07-19 20:23:34 +00:00
drh
5e4d77fa88 Describe JSON parser performance testing procedure in test/json/README.md 
FossilOrigin-Name: c47056e158073ee746c233dd2729cf224ea9e38821b15aa71ebf97607e602281
 2023-07-19 19:23:30 +00:00
dan
d1fbaa071b Fix various issues with code added to this branch. 
FossilOrigin-Name: 8d09011fa2c6ae9cc88e1766f9aad4578efbf9e0e311b8c6efdffe7a3f88f923
 2023-07-19 18:47:02 +00:00
stephan
55f318e53f More internal refactoring and docs for opfs-sahpool. 
FossilOrigin-Name: 64ccf6177a019eab46fb3345ad1e8ba80eaf2c9da55767031f9f04ccd16afb4d
 2023-07-19 17:47:02 +00:00
stephan
96cb7007a9 Update the development-over-ssh docs for the wasm build. 
FossilOrigin-Name: 500109bd0a4c134b91c37f397ff1ee828e09c17f7ecd153f975ede748caee7bb
 2023-07-19 17:46:28 +00:00
drh
50834d8b74 Further improvement to JSON parser performance. 
FossilOrigin-Name: 144c8ccf6e5bb2527dd98742f0d67e0a16c627e7c67f754ce8ed4c4fb5b8d8b6
 2023-07-19 17:24:36 +00:00
drh
8376ae7295 Further performance enhancements to JSON parsing and rendering. Total 
performance gain for large JSONs so far on this branch is about 11%.

FossilOrigin-Name: adb4d6b007cbe9d7c9670f5fc196443ebe0f3a89df1f3290ba6247fcf83fe5bd
 2023-07-19 15:06:29 +00:00
drh
7a816e771a Performance optimization for parsing large JSONs that contain lots of text. 
FossilOrigin-Name: c9fbe0185cd5d64950724b00cd0bfb3a7939a985040465a0f35f445acb6e94a6
 2023-07-19 13:50:31 +00:00
stephan
eafb4418a0 Major restructuring of opfs-sahpool to improve maintainability and hopefully make it easier to reuse these pieces in the upcoming JSPI-based VFS experiment. 
FossilOrigin-Name: 534481cd0c2e6f62dd0a82f25d4b78fdcc671eb70d6966693c98212a6420891c
 2023-07-19 13:31:29 +00:00
stephan
cd302a5e32 Appearance and legibility improvements for tester1.js. 
FossilOrigin-Name: 4fd3ed1b6405f690825c9993e56b199d4a7fb497cf9131815c803150b8b96528
 2023-07-19 11:33:52 +00:00
drh
37fd50df1f Two minor improvements to sum(), one of which was inspired by 
[forum:/forumpost/af5be98dbc|forum post af5be98dbc].

FossilOrigin-Name: a0d3e7571aded8d1e03908059d2d5aa5d62ec49bff099cb38f6f35df5e4b18b5
 2023-07-19 09:52:10 +00:00
stephan
780bc4c557 Internal refactoring in opfs-sahpool. Move OPFS tests to the end of tester1.c-cpp.js. 
FossilOrigin-Name: 6bd5a7413dd830ca41b587a2826fb599a2196fb0186646a2333500f950b3cf4d
 2023-07-19 08:18:25 +00:00
drh
9246c85b61 Do not use the [/info/609fbb94b8f01d67|viewscan optimization] on a query 
that has only a single loop, as the cost adjustments can cause problems
for outer queries.  Proposed fix for the performance regression reported by
[forum:/forumpost/64d36440e473516c|forum post 64d36440e473516c].

FossilOrigin-Name: 76152ad2ffe56034f2fd93d9a1ce9358e1677a7e9cd3dcd9f3a34a5c956a463e
 2023-07-18 21:06:19 +00:00
drh
6a9be2cbc7 Improved ".wheretrace" for the VIEWSCAN optimization. 
FossilOrigin-Name: 27057ea76b5f72c73fb6f16094736685019643c665b49fd0bb8d60a812ce2338
 2023-07-18 20:41:09 +00:00
dan
d05bf0fe61 Fix various problems with fts5 contentless_delete=1 tables. 
FossilOrigin-Name: 0d005112b8aca9e9eca9d86d5fed9168f6a0218fd290b5489b9e7b05714610f4
 2023-07-18 19:52:32 +00:00
stephan
aed5abcc0f More opfs-sahpool tests. 
FossilOrigin-Name: 60713fa9c4627ef17e0b8778eee37913d2b930c5a06414721a00af30e1395090
 2023-07-18 19:47:19 +00:00
stephan
0649a1a05d Fix a shadowed var in opfs-sahpool and add more tests. 
FossilOrigin-Name: 9c8a73aff0f291e0c18072372e0d8961d3a05910489598d0d342227d99871954
 2023-07-18 19:27:11 +00:00
stephan
b0dd9d427f Further refactoring of opfs-sahpool and start integrating it into tester1.c-pp.js. 
FossilOrigin-Name: 91c789234963b660ae900f0738906b28a477993709e286d8125b2f4d6101601c
 2023-07-18 18:52:41 +00:00
drh
3f47e7551c Merge the FTS5 memory leak fix from trunk. 
FossilOrigin-Name: fb65cb73d7ea22a8b20dccfa3abdaaa809eee4fcee6fe4846bd2e598ceb49aa4
 2023-07-18 17:43:47 +00:00
dan
a5b1f34125 Fix memory leak in fts5 introduced by [def41225]. 
FossilOrigin-Name: 4dcad2db743fdb9ef72871ca5a4d1384f76cb697161b0f5110e2670a83a18e8a
 2023-07-18 16:29:34 +00:00
stephan
d69e6e6efb Major restructuring of the opfs-sahpool bits to better support multiple instances per app (each sandboxed from each other). Eliminate the pesky promise resolution race condition when two such instances are loaded in parallel. 
FossilOrigin-Name: 95e5fa498f71708caeb3394636c4853530a8b2d54406e503f32750732d6815d5
 2023-07-18 16:24:51 +00:00
dan
cb1d6b997e Merge trunk changes with this branch. 
FossilOrigin-Name: 5f66eb4e2603278dcc9dbfe4bf506cba1aa03180cfb492a0dfc3a8be32cc994b
 2023-07-18 13:55:55 +00:00
dan
263a8e8c0a Fix a buffer overrun in fts5 that could occur in 'secure-delete' mode when dealing with corrupt database records. 
FossilOrigin-Name: def41225b2f10b0294ab506f28bc87873688d4030f7056839298ff18e058ece5
 2023-07-18 13:55:19 +00:00
stephan
dec4cea24c Refactor opfs-sahpool to support multiple instances, each with a separate VFS name and directory. 
FossilOrigin-Name: d036eaf6ac60c576428db40f015733c5d5425f7d613194fd8d9d4d98659077c4
 2023-07-18 12:09:16 +00:00
dan
2cccced140 Fix cases where a row is inserted into a contentless_delete=1 fts5 table and then deleted within the same transaction. 
FossilOrigin-Name: d928856a226fb7f001e55ff7e8eb58a656b982f1efa811de46c382b8b7cd778c
 2023-07-17 18:40:39 +00:00
dan
55e0fd4a9d Do not allow the 'delete' command to be used on contentless_delete=1 fts5 tables. 
FossilOrigin-Name: cc694b83408ccb5d42204cb624145c76e95329cbe1d1fe8815c70a7a00af231a
 2023-07-17 17:59:58 +00:00
dan
24433bea19 Merge trunk changes into this branch. 
FossilOrigin-Name: c4fb2f2ea0afe638fd7cffd89fbdb0a91589577c6f8299c7bbc17ac121be518b
 2023-07-17 11:47:42 +00:00
stephan
99d4a2db46 Another -DSQLITE_TEMP_STORE=3 to 2 change for the wasm build. 
FossilOrigin-Name: 5ad8f9987c11b5db67edd69a3471ff388140d6268d5d3f5a05dec6bb6d92ac05
 2023-07-17 07:43:04 +00:00
stephan
ed640943c4 sahpool vfs: rename defaultCapacity to initialCapacity, per feedback. 
FossilOrigin-Name: 2c9c95d087c7db2a975ffae15af5ade621411c493145b889f0c9157a55c4952a
 2023-07-17 07:14:12 +00:00
stephan
2bf2ff605f Change wasm build's -DSQLITE_TEMP_STORE=3 to -DSQLITE_TEMP_STORE=2, primarily so that clients can optionally shift temp files from memory to OPFS storage. 
FossilOrigin-Name: 764430e804332b8f7b2f63a3c0c8acf8f6cc92534858d89de2d310938c1c0d27
 2023-07-17 07:13:44 +00:00
stephan
039081123c Remove an extraneous JS script import. 
FossilOrigin-Name: 5a205b25912f3feea594a2161a4b8f3955bd29163b39cb0e621b3abdb81fc24d
 2023-07-16 21:07:01 +00:00
stephan
8d67d501e3 Add sqlite3-vfs-opfs-sahpool.js to ext/wasm/api/README.md. 
FossilOrigin-Name: 58d05201042804c682a417495f13ab47a535e353da6cc307f9d726a289e71ca2
 2023-07-16 20:42:19 +00:00
stephan
1ff2903785 Minor doc tweaks. 
FossilOrigin-Name: 0cb0d7a9e73e48406f146ebebf211311fd0a0b14b285f0af9e2b645c3d2f172a
 2023-07-16 20:20:03 +00:00
stephan
d703d25a35 Correct a missing assignment. Minor doc additions. 
FossilOrigin-Name: 1c4957d0ef23ff14d4f7bfb33a809dd92712ee9faf77b6052f823eb55de15cf6
 2023-07-16 19:29:50 +00:00
stephan
a34f27daed Doc tweaks only, no code changes. 
FossilOrigin-Name: 044c28dffd45f7c4484686995edd4a1b92151450743968e7d0f662b5c850aa6b
 2023-07-16 19:20:45 +00:00
stephan
24873723f4 Add a way to remove the opfs-sahpool's persistent state from OPFS or unlink() an individual file. Doc cleanups. 
FossilOrigin-Name: 80982daac3c098033dbc249bb7a17ef84ae218d2d789f8644e7f4af18b553d24
 2023-07-16 17:51:43 +00:00
stephan
da6a42a921 Move SAH pool configuration options from the library-level config to a config passed to the VFS install routine. Extend and document the PoolUtil object. 
FossilOrigin-Name: d2ed99556fa1f40994c1c6bd90d1d5733bebc824b1ebfabe978fae9e18948437
 2023-07-16 16:52:09 +00:00
stephan
d62c464541 speedtest1.js: only install opfs-sahpool if it's provided via --vfs flag, to avoid locking errors in concurrent speedtest1 tabs with other VFSes. Add opfs-sahpool reserveMinimumCapacity(). 
FossilOrigin-Name: aa94c8abfbdfc4c7b36554c4b3ea90a5065e7e3f4294c64c8cbf688b4688300d
 2023-07-16 14:07:59 +00:00
stephan
aa1b2a80ee During "opfs" VFS init, check for URL param opfs-disable and, if set, do not install the VFS. Added per forum suggestion to provide a way to help rule out misinteraction between the "opfs" and "opfs-sahpool" VFSes. 
FossilOrigin-Name: 29905b7a75b73e32125bf9116033cae7235a135b668a3b783a3d8dcb0bc80374
 2023-07-16 12:02:37 +00:00
stephan
8449dacbd4 Micro-optimizations in opfs-sahpool. 
FossilOrigin-Name: 52f23db948ae9694df69c00177b85cb569e9b211350a4a2dbf249e7cd8de700c
 2023-07-16 11:49:18 +00:00
stephan
d0ae50411f Redefine what the opfs-sahpool installation promise resolves to. Fix addCapacity(). Add utility methods to import/export files. 
FossilOrigin-Name: 809c6f4de3653ad7a7751af45a7a0d6cb20c3ee3be80c69833c729242227d970
 2023-07-16 10:02:41 +00:00
stephan
28d46cce0b Minor cleanups in the opfs-sahpool VFS. 
FossilOrigin-Name: 279e09070918dab7b60c39179ebb7eb931ca6bd4e589b414f436740499a2f910
 2023-07-15 21:08:48 +00:00
stephan
38d1db9b79 Correct opfs-sahpool VFS after the pebkac involving the previous speedtest1 runs. Make that VFS explicitly opt-in to avoid certain unfortunate locking situations. 
FossilOrigin-Name: 41bf1fe31f2f3d0daa2bac25dc57262a4b90f22fed6fa97e4e92467c32ae02dc
 2023-07-15 19:08:58 +00:00
dan
0dac350f3c Store large tombstone hash tables in multiple database records. Ensure the same hash tables handle rowid 0. 
FossilOrigin-Name: 4410e60d0c76e057ee962124f9239c6e17fd5ccafdbb4d9b703448eabd7781e3
 2023-07-15 18:57:57 +00:00
stephan
eadabc6513 speedtest1 JS: only add --memdb flag by default if no --vfs is provided. 
FossilOrigin-Name: 676ffe6280c1ce787b04d0cdb4a0664229c6125c601af4b18d1bfa125aac3675
 2023-07-15 16:30:46 +00:00
stephan
bb65feb869 Rename opfs-sahpool removeCapacity() to reduceCapacity(). 
FossilOrigin-Name: fff68e9f25a57045e9d636b02ffa073cf1b984b2587d4fce10f6e35c9988469c
 2023-07-15 11:37:34 +00:00
stephan
88af76f62d opfs-sahpool VFS now seems to work, in that it runs fine (and blazingly fast) in speedtest1. Add sqlite3.config options for the high-level configurable parts of opfs-sahpool. 
FossilOrigin-Name: 5d92d5f4d8ac4cfa37ba473e5cc861628b783bbf1ae4d138bcae8b9d6cc6e798
 2023-07-15 11:23:57 +00:00
stephan
100bc4429a More work on porting the sahpool bits. Revert to random backing-store names because it works better with the capacity-reduction algorithm. 
FossilOrigin-Name: b4e005fd38b06b8d2f2317b955b93807e80a6a18db5f06d7747978d3bfa11411
 2023-07-15 01:02:38 +00:00
stephan
bee3213145 Switch opfs-sahpool to use deterministic backing-store file names. Delay VFS registration until after the pool's files are all opened. Fail vfs init if the client's OPFS API has an async FileSystemSyncAccessHandle.close() method (indicating that it's outdated). 
FossilOrigin-Name: 7c6697ededee9a64962ac6fd78934c6d6e39258b9558a03c1a6c02bf3be1759e
 2023-07-14 23:02:58 +00:00
stephan
92bf6fd4f6 Integrate the opfs-sahpool VFS into the JS build and get it loading (but it's still far from functional). 
FossilOrigin-Name: ef96e6b586825a2b3ed011174309cba8ce1031876c86dc59ed87ab9bbc64f57f
 2023-07-14 21:48:45 +00:00
stephan
50c61825fc Initial sketches for an alternate OPFS VFS which uses a pool of pre-opened SyncAccessHandles to bypass the need for a dedicated I/O worker and the COOP/COEP HTTP response headers. Currently completely non-functional. 
FossilOrigin-Name: a93de9f2a553a3a4edd1b361dd6f465a1b0b5b51f7bb8ede432067aedcfefda4
 2023-07-14 21:17:29 +00:00
stephan
a02d6d8237 Minor internal cleanups in the OPFS VFS. 
FossilOrigin-Name: 984d491eb3fe06f714bf07d6873321f3992a072812b46508e599bfefd39dff3e
 2023-07-14 21:06:00 +00:00
stephan
8ba619b5f5 Minor wasm build cleanups. 
FossilOrigin-Name: 816b503f093c4e6d92d0eb2f9fbd841acd01cc9bc89ee58d961b56c64f71406a
 2023-07-14 12:21:05 +00:00
stephan
f4449dc0ba Reintroduce WASMFS+OPFS support, albeit as a separate (less portable) WASM build. 
FossilOrigin-Name: 7e4c594e9591101f6b727212c99f561498444e86d3d059e5864a717c614f903a
 2023-07-13 18:35:48 +00:00
stephan
d655290c99 Minor cleanups related to reintroducing wasmfs support. 
FossilOrigin-Name: 6918a927b5ecd1df5916b70d52d8d5ce289869c3f323e577b7dfc7c939084400
 2023-07-13 18:13:52 +00:00
stephan
bb9549e0be More work on the wasmfs build and its test apps. 
FossilOrigin-Name: 953b8557194e9451dcf9f3bb433eafb67961487325e6519e675e73fecf850bfb
 2023-07-13 17:27:05 +00:00
stephan
d51cefd1bb Get speedtest1 wasmfs variant running again. 
FossilOrigin-Name: 0f77e031daf0121f7603409cedf57f138edbb43083cbbcf00d2a423e16de25a7
 2023-07-13 16:33:45 +00:00
drh
bc6d949aa3 Fix an off-by-one error in tests for cells overflowing the end pages. The 
error is completely harmless for the default page cache, but might cause
problems for application-defined page caches that pack pages tightly
together.

FossilOrigin-Name: ce6793e954f291b6f5c29175baf730ce217328de1f0601b8935daac62af5f448
 2023-07-13 14:49:39 +00:00
stephan
cc336321f8 Significant surgery on the wasm build on the way to incorporating wasmfs as a first-class build option. 
FossilOrigin-Name: 215c37fce38cf647e073480689b79d952af7eb8165ae08c7f5caed27003fecfc
 2023-07-13 14:08:30 +00:00
stephan
d41d454fc1 More work on the wasmfs build. Resolve the inconsistent argument type passed to sqlite3InitModule() in such builds. 
FossilOrigin-Name: 4850a9e2d2b41b91e91b9ef99a6d26ddf11a161b4c970fc00d2d095606729a04
 2023-07-13 10:41:41 +00:00
stephan
0f0e5700ba Minor cleanups in the wasmfs build. 
FossilOrigin-Name: f64477f21040c265061ad9b7f601e74159fd6141e7e4b122c80bf244ab6ddb76
 2023-07-13 04:54:18 +00:00
stephan
01bdff7a1c Get wasmfs build and its bare-bones test app working again, albeit currently in ES6 mode only. 
FossilOrigin-Name: 647761ed422f196f94facc88bbddd7219a2c1a6301a5f847b0a32d3e405233a7
 2023-07-13 04:26:13 +00:00
stephan
81602595a0 Replace 'self' with 'globalThis' in some JS test code. 
FossilOrigin-Name: 53eeffc9b1ea8c3ee7d7cdcc220997ed893403e45b35d8ea10805ea3b0435587
 2023-07-13 03:46:43 +00:00
stephan
5da49ff562 JS: change the default OPFS VFS journal mode from PERSIST to DELETE, as DELETE is faster in current browsers. 
FossilOrigin-Name: a061a43e36b8f3cb7c465fd2e54cfcd9dceb07e232ee78602bbdd1818acd05ea
 2023-07-13 01:51:16 +00:00
stephan
a5512fb179 Remove obsolete ext/wasm/api/sqlite3-wasi.h. 
FossilOrigin-Name: f661a6e0c1ac081ee402e64590eabd427440db1b717be9166bdfc025d0b0d899
 2023-07-13 01:48:53 +00:00
dan
fc158d3870 Avoid a case of an infinite loop in fts5 when dealing with corrupt records. 
FossilOrigin-Name: 0e801f11cd2d50fc710a80c2b3b805c7801e660cff1bcc87be89133d43796524
 2023-07-12 20:24:23 +00:00
dan
9f6c0d0027 Fix a crash that may occur when handling corrupt records in contentless_delete=1 mode. 
FossilOrigin-Name: 74d7610a8e32ac62a608141ff16bfe517c8d2cdfd43f81ef3a5df2b2003e541a
 2023-07-12 19:34:32 +00:00
drh
9faa3bbdbb Get access to SQL functions and virtual tables used for debugging and analysis 
of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST.

FossilOrigin-Name: 383de8e2259adb3f8881a618b8b5e06c1f3b2bd7ac274bd047410ad7d2a18f0f
 2023-07-12 18:38:47 +00:00
stephan
b3b64179df Internal JS doc fix. 
FossilOrigin-Name: 7c5ea02205a360a11fa9043be12c678eb7fe6f6a0b63b09a3dc976beae881e3f
 2023-07-12 09:30:13 +00:00
stephan
a17b04f840 Correct handling of the speedtest1 --without-rowid flag in the HTML front-end to speedtest1.wasm. Reduce default speedtest1-with-OPFS workload size in ext/wasm/index.html to speed up manual testing. 
FossilOrigin-Name: 2daba2cdd3890c87899ff3eb0c1706c814f0ae62e72822eef3b0d4a885679a6b
 2023-07-12 09:16:18 +00:00
dan
b184c91076 Have contentless_delete=1 tables support REPLACE statements. 
FossilOrigin-Name: 2f553a660e00564e51bc4209c92bd3628fb1266f4a52832792fbf91e4234a0ba
 2023-07-11 20:57:19 +00:00
dan
b92669db73 Have contentless_delete=1 tables support regular DELETE statements, instead of just the special INSERT syntax. 
FossilOrigin-Name: fffb8616905501669a94231d5d9f53446bf09553353f2cdab7c43ca54bbb7fa6
 2023-07-11 20:19:46 +00:00
drh
8a1104940d Reduce divergence between branches: trunk and wal-shm-exceptions. 
FossilOrigin-Name: d38ed6b1fc6b572997d520c36f256c11404fb1bd5f28acab07466bfc96554a8f
 2023-07-11 19:54:17 +00:00
dan
d23f210377 Use a hash-table instead of a flat list to store tombstone rowids. 
FossilOrigin-Name: 948267b066d0dbe667881b3d26a007fa24576da6e57c112676fadeb846c13f0b
 2023-07-11 18:55:19 +00:00
dan
6788c7b7c0 Begin adding support for deleting rows from contentless fts5 tables. 
FossilOrigin-Name: e513bea84dfaf2280f7429c9a528b3a1354a46c36e58ab178ca45478975634e0
 2023-07-10 20:44:09 +00:00
drh
4c40b7b819 Always use the "LL" suffix on 64-bit integer literals. 
FossilOrigin-Name: 07d95ed60f0a17ea13b4bc19c2ab2ec9052fedd27c9e1e57a1ec6e3a6470e5b7
 2023-07-08 17:42:24 +00:00
drh
bc532ae15f Fix harmless scan-build warnings. 
FossilOrigin-Name: beab3c98639be531744e60440223bb9ee76bc15234aff05e5efb273c8241dfd8
 2023-07-08 14:27:55 +00:00
drh
50ba4e3efd Omit the use of memmove() in sqlite3FpDecode() in an attempt to avoid 
spurious warnings from valgrind.  Also makes the code slightly smaller and
faster.

FossilOrigin-Name: b409943af00e35ad05906d0a80ffa1225a66f3d807c131e00cfbf6671a03a981
 2023-07-07 18:49:08 +00:00
drh
afef7fc136 For the "Valgrind" test scenario, set LONGDOUBLE_TYPE=double as valgrind 
does not correctly emulated extended precision on x64.

FossilOrigin-Name: d3532eaed1cc671d8149b1bd34ead2a88fb83502a2898a5f60aea0a7daf18958
 2023-07-07 12:18:26 +00:00
drh
bae91082dd Add option 2 to SQLITE_TESTCTRL_USELONGDOUBLE to move the setting to its 
defaults.  Enhance the CLI and testfixture to make use of this value.
Base the default on sizeof(LONGDOUBLE_TYPE) rather than on sizeof(long double).

FossilOrigin-Name: 64e6bd1c25d8e1dbfe511cba1921ff052c0fa4fe410fc9ce4435700a70cb88b1
 2023-07-07 12:08:22 +00:00
drh
7d0103bb06 Improved handling of integer inputs to the new sum() implementation. 
FossilOrigin-Name: 31561c63bde6af2210296810fd66232b935033b4d26273a6b8a52142c54b4e12
 2023-07-06 20:34:06 +00:00
drh
68c88ea04f A more precise implementation of sqlite3RealToI64(). 
FossilOrigin-Name: 8813317cb52990d90bd48de163b6e35ddcad2b6f3187aa28b28e2e088b4489ce
 2023-07-06 18:29:15 +00:00
drh
3e1e7aad3d Fix octet_length() test cases so that they work when the database encoding 
is utf16.

FossilOrigin-Name: ad2e39626e6115fd322ef9495ae3bcd91343d7131a725f1bf24211a51a21dcc2
 2023-07-06 17:14:47 +00:00
drh
59f13119cf Use the Kahan-Babushka-Neumaier algorithm to improve the accuracy of sum(). 
FossilOrigin-Name: c63e26e705f5e967e14ef6aea8ce226548293ad8d25066069f29fa89673913d2
 2023-07-06 15:44:38 +00:00
drh
26cd8bc1af Fix minor inaccuracies in the new sum() algorithm. 
FossilOrigin-Name: e3f7a960c9bd8e84cd70f0585bb955d043604a92001d0e2bf6c1216bb1fd7221
 2023-07-06 14:45:53 +00:00
drh
45d7562843 Experimental attempt to boost the accuracy of sum() using the 
Kahan-Babuska-Neumaier algorithm.

FossilOrigin-Name: ebc5edd3b10c1102b07b9fb0d6837266b81e55504ef883b9b8a7ad5e8ab29dd2
 2023-07-06 13:19:10 +00:00
drh
3748b7329f Use "volatile" isntead of "#pragma" to get floating point calculations 
working correctly when compiling with GCC for x86 machines.

FossilOrigin-Name: 9427f42687ed6d97c474bf42d0c3e82d6f4b0075e74206adcb5699d72e32140e
 2023-07-06 00:55:06 +00:00
drh
efd0cf87df This is alternative work-around for the x86 float-point problem that uses 
"volatile" rather than compile-specific #pragmas.

FossilOrigin-Name: 1d972a690fdc70ab40862bd38427d68b48e8802ddf8e5c301f2d58ce2178b6ec
 2023-07-05 22:05:18 +00:00
drh
728650ecb3 Improved comments on the work-around to the GCC x86 floating point wonkiness. 
FossilOrigin-Name: 7b4c16731e7bf6f03f5adf4fcb2008c0b19be473fb1b90b405c217c08916586a
 2023-07-05 19:56:14 +00:00
drh
e68899f1e4 Work around for an apparent GCC bug. See 
[forum:/info/ee7278611394034c|forum post ee7278611394034c] for details.

FossilOrigin-Name: 5d9e9364808793d65925d4efbfde0f4246df258758f15e8ce1105070d1018fe6
 2023-07-05 18:59:52 +00:00
drh
85ca6d7ace Make the order of operations explicit for the error term of Dekker 
double-precision multiply.

FossilOrigin-Name: 28f57b34e6b11184a36e363a985e7531bddd7be48a5e039a670e5acf748eedda
 2023-07-05 15:34:30 +00:00
drh
1790ccb98b Simplification of the dekkerMul2() subroutine. 
FossilOrigin-Name: bfe6432a46fd41bfadf4516a73b4e2523d9b453c3e4486fe6df7e1d25275dfb3
 2023-07-05 14:42:50 +00:00
drh
6aa5a42beb Add the ieee754_inc(r,N) function to the ieee754.c extension. 
FossilOrigin-Name: 9a36f269451d8832f68e223ac3995493704364e7fd0cc772989c96e863a08d29
 2023-07-05 12:00:32 +00:00
drh
7623795018 Enhance the sqlite3AtoF() function so that it can operate with or without 
long double and provides accurate results in both cases.

FossilOrigin-Name: fd987508d5589d42b46625c97cfb2feeac70fdb1cb11224610554b07a60644e6
 2023-07-03 18:32:29 +00:00
drh
aa4356da68 Enhance sqlite3AtoF() so that it honors the USELONGDOUBLE test-control. 
Enable the test/atof1.test module on non-intel platforms.

FossilOrigin-Name: b7850fd480535980d66f107c2746e7f7fc0a5379ec8244d84f7785533c83f276
 2023-07-03 18:18:35 +00:00
drh
930e4167bd Store the fp-speed-1.c test program used to measure relative performance of 
SQLite versus the standard library, in the test/ directory.  A header comment
explains how to use it.

FossilOrigin-Name: eaff376277edceb043e9aac9d2df3e1157f576605508733be8a0a0aa4d720e1d
 2023-07-03 15:30:50 +00:00
drh
521f20e3d3 Add the --nolongdouble option to speedtest1.c. 
FossilOrigin-Name: 6bf6018a00d61c5f0e53b37b00befaf5ad1d85323f41cf0cea25d8c6c1017096
 2023-07-03 15:11:28 +00:00
drh
c9f240cea4 Improved precision of binary-to-decimal conversion of floating-point values 
on all systems but especially non-x86 systems that lake extended precision
hardward floating point.

FossilOrigin-Name: 32befb224b254639e756fd493f300507ff60c5222e00a7338dbcd39140133964
 2023-07-03 15:03:59 +00:00
drh
1b3c6e727e Whether or not to use the long-double floating point conversion technique 
is now a test-control setting.

FossilOrigin-Name: 02ae6caff1f7925c696ad263fe78b1036364d24b6373e1baec10d53aafb14a12
 2023-07-03 14:56:49 +00:00
drh
0478d1d88a New test cases for floating-point conversions. 
FossilOrigin-Name: 9435ed76cd9bb1be02768b3853a2323834f67bda75d3e3f76f6aa7bd0bade3d7
 2023-07-03 12:58:18 +00:00
drh
b884158434 Use Dekker double-precision floating point to do the value normalization 
step in sqlite3FpDecode() for systems on which "long double" is 8 bytes or
less.

FossilOrigin-Name: 51f492b440728151ce64168d5436ba87ca60d84b89408e47c593fa731355afe4
 2023-07-03 11:44:15 +00:00
drh
841e596ad0 Final clean-up of the idea of using Dekker double-precision floating 
computations to improve accuracy.

FossilOrigin-Name: 40285d631072b79aba39cc20d4b217bdf0069f29ac9902cb97dec28d3e4cde11
 2023-07-03 11:39:38 +00:00
drh
bc2b65832e Squeeze as many bits as we can out of normalization. 
FossilOrigin-Name: 25d1ad6df1ed460c2cc9f32e8a5106e121ae502044f83abf431c1d5b3879eb56
 2023-07-03 10:35:29 +00:00
drh
5270d7424d Faster Dekker multiplication that removes the restriction on input magnitude. 
FossilOrigin-Name: 2994caf5884be07c889519c78fbac4ddcf267fcfe6a3265ecb6390bcd574532e
 2023-07-03 10:18:02 +00:00
drh
37b188fbca Fix one constant in the normalization logic. Improved error output 
from atof1.test.

FossilOrigin-Name: d3c48807100a358a70fdd799c8935eba1b765ace2e1ddea4475fd673006cb6da
 2023-07-03 10:00:38 +00:00
drh
c8885d42fe Replace the dodgy error estimating logic in the previous check-in with 
full-up Dekker double-double multiplication, and this idea works much better.
There are still a few inaccuracies, but it is much closer.

FossilOrigin-Name: 4fa6938dac2d3d813a37664053db31451a2a065f78dd212488f5f7f8d583ddc5
 2023-07-03 09:07:23 +00:00
drh
c27bda0b79 An idea for improving accuracy of fp-to-decimal conversion for systems that 
do not have "long double".  Does not quite work.  Retained only for
documentation purposes.

FossilOrigin-Name: 702243333843219f8904ee0fd12283080d6c6b3fc1ffb36c534cdefb3563c00d
 2023-07-03 00:40:37 +00:00
drh
fecdff4354 This check-in adds the use of "long double" to sqlite3FpDecode() which when 
run on x86 using a compiler that translates "long double" into binary80,
causes all legacy tests to pass.  But on a system where "long double" is just
an alias for "double", some of the tests are still failing.

FossilOrigin-Name: ce06982f880339cf46704e95c907249827c3e44af2b9420005200ca8abd3f371
 2023-07-01 20:21:27 +00:00
drh
453be55b85 Reinstate the fpdecode() SQL function for testing, but only when compiled 
with SQLITE_DEBUG.

FossilOrigin-Name: 07eab52e0801bb0e4743b304a06ad16e18cdf8eaa18c0faf8d47a1f5d8576ea0
 2023-07-01 18:33:26 +00:00
drh
17c20bb15e Improved rounding policy. 
FossilOrigin-Name: 6f1122e942b8269552daaf13d647d200d8546ec25f36310d67037c6b58d09976
 2023-07-01 17:56:00 +00:00
drh
9ee9444a0a Improved response to Infinity and NaN. Remove the termporary fpdecode() 
function.

FossilOrigin-Name: 76ab8ae809a47a66688e2d50c20dc87ce946d82e9ffebb3adda55c451fad07fc
 2023-07-01 15:23:24 +00:00
drh
42d042e602 Progress towards fixing corner-case floating-point formatting issues. 
FossilOrigin-Name: 1138b8d16edef16d2943d407e289d07d546a14e64c3efa3c94127631e06fafeb
 2023-07-01 14:03:50 +00:00
drh
3f073f9ef8 Fix an issue with the "+" attribute in printf(). 
FossilOrigin-Name: bdfd1eb1d7b4c570a53d58789dac0bf55fdaf8eba515d5a0caf2e57e25b51952
 2023-06-30 23:42:46 +00:00
drh
aebeabacff Begin cutting over sqlite3FpDecode() into printf(). The code in this 
check-in gets a lot right, but there are still some formatting
discrepencies.

FossilOrigin-Name: c134d423e79ba5d3343d7480229822a0c83210f67f12129e0237181f892f97a5
 2023-06-30 23:18:44 +00:00
drh
002330dc48 Add decimal rounding to the sqlite3FpDecode() routine. 
FossilOrigin-Name: 27871140caa833f0bc0962e44356993938e93dcf81c1074382b1560a3e1aeb61
 2023-06-30 19:41:57 +00:00
dan
f8be243fef Add experimental ".scanstats vm" command to the shell tool. 
FossilOrigin-Name: e727640fb5c17d23b8e27972065b4acbf169c9240298d3ff7aed092b727d052d
 2023-06-30 19:14:35 +00:00
dan
e6d7ae24e3 Fix an error in the previous commit on this branch. 
FossilOrigin-Name: 47c11ca90f98ffc4d91f07e2ab35826a604a2c903008eeddf8b814bb984971f2
 2023-06-30 19:13:27 +00:00
dan
aec0aa83a9 Add the "nexec" and "ncycle" columns to the bytecode virtual table. For accessing counters collected when SQLITE_DBCONFIG_STMT_SCANSTATUS is enabled. 
FossilOrigin-Name: f7b163a319bee9e935a4abf0bd87e16c9974cba5ae75b0cbba63c9da168f2006
 2023-06-30 18:59:10 +00:00
drh
a1b0ff1735 Experiments with a new algorithm for converting ieee-754 binary64 numbers 
into decimal.

FossilOrigin-Name: e923405e448385085224f9289991b303d86b02763535ea77d6fcee98ba6fc1f2
 2023-06-30 18:35:43 +00:00
dan
24b368da8d Add support for sqlite3_stmt_scanstatus_v2() profiling of GROUP BY clauses that use a temp b-tree, and for sub-queries implemented as co-routines. 
FossilOrigin-Name: 7afad1f759f7ceda873c6d869422fd56fe4399c2d24d47ad9bc3b84b06d830d1
 2023-06-30 18:31:37 +00:00
dan
4cd3a59a79 Improve support for sqlite3_stmt_scanstatus_v2() profiling of sub-queries implemented as co-routines. 
FossilOrigin-Name: 4e8718dc35dbbaf75f17265a88d14acd9750dc75efbadf41377f9c97e732009c
 2023-06-30 18:23:53 +00:00
dan
07d76f7b95 Add support for sqlite3_stmt_scanstatus_v2() profiling of GROUP BY clauses that use a temp b-tree. 
FossilOrigin-Name: 796eadcc50e7ea9ec348b1f7e2d6a5a3de78c675a07f665078309e6d81479e7d
 2023-06-30 17:14:36 +00:00
dan
1bfe92a691 Add extra debugging code for sqlite3_stmt_scanstatus_v2() to test1.c. 
FossilOrigin-Name: f936f101406069b29218c89a36581b4497226fb61906782ea368f12d943c901c
 2023-06-30 16:16:51 +00:00
drh
48114d08cf Completely unwind the enhanced precision sum() from [a915f15a916af698] so 
as not to offend UBSAN and OSS-Fuzz.

FossilOrigin-Name: 85be05b66ef975f02a3e7b2984bcab97d9280c7f3b6ee1e16718de549f240d46
 2023-06-30 14:01:09 +00:00
drh
dd8b12a6c5 Omit the doubleToReal() function in vdbemem.c. Use the 
equivalent sqlite3RealToI64() function in its place.

FossilOrigin-Name: 625820e8ebfdcf513c81b1b632bbe2aa882a8fbba52001113dc6f56270fa5ceb
 2023-06-30 12:59:06 +00:00