mirrors/sqlite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
25,002 Commits 1,487 Branches 166 Tags 416 MiB
de6186e04f
Commit Graph

3443 Commits

Author SHA1 Message Date
dan
88a0637f57 Ensure the page-size, auto-vacuum and other settings are copied into the recovered database. 
FossilOrigin-Name: 078520f2825bcb82d01b30209409e9bf8b6c99084c192345df41c9f531d08f51
 2022-10-06 21:00:23 +00:00
dan
4df12ce5a8 Add code to determine the database page-size by searching for well-formed pages. 
FossilOrigin-Name: 0dbd0ccef532e076fa82c9fad4dc9fe632c0ef43e3483cc288cbb7eca336a9b9
 2022-10-06 17:20:12 +00:00
dan
adc686043e Improve handling of corruption in the sqlite_schema table b-tree structure. 
FossilOrigin-Name: 2be0dba12fb9b98cd9632f1ff3f30aeb29661a2e121b68c2f3335617b027797b
 2022-10-05 16:58:28 +00:00
dan
461fa44be0 Merge latest trunk changes. 
FossilOrigin-Name: b0feecaa84b88f534edb49056cf7e92256f4a687c8b4c44e1d1186709a304378
 2022-10-04 19:47:17 +00:00
dan
041cc06139 Better handling for databases with corrupt headers. 
FossilOrigin-Name: 17f68d803685405d880025134c71bac01077962d677f2902a608952d0b9cdb5a
 2022-10-04 19:43:12 +00:00
stephan
9a55773b2f Replace time-based auto-unlock of opfs sync handles with lock acquisition/release via sqlite3_io_methods::xLock/xUnlock(). 
FossilOrigin-Name: 2625b7cfe1640c1d7e779ec1f37db970541598c0dc3e22e5eecf3c772d95ad40
 2022-10-04 17:06:51 +00:00
stephan
ed3182f233 Tweaks to the opfs async wait/relinquish times. 
FossilOrigin-Name: 35f33c23e5849de1c43c4499ee0a7fa11d26ae34949c1e820c3fa8e8873f9c2b
 2022-10-04 11:14:23 +00:00
stephan
98e120fca0 Update ext/wasm/api/README.md to account for recent changes. 
FossilOrigin-Name: 9d488081fc82a9abe3e81c7897fe6b29b6b337f62d0c62cb9cec7517bd54b53f
 2022-10-04 09:12:43 +00:00
stephan
c7fb48d4b6 OPFS async proxy: add a wait-and-retry policy to the get-sync-handle step to help account for cross-tab locking. 
FossilOrigin-Name: 45c48c63d311052105d102189208495b2b53fa3c4174884ecf63b757aa4016e5
 2022-10-04 09:12:05 +00:00
stephan
f6b6188bdd Update the 'start in speedtest kvvfs' link in speedtest1-worker to account for recent changes. 
FossilOrigin-Name: f4d5ba412e0c1a031430a32c9ed1e11a09af33f9883150e660cc05e49fe56e6a
 2022-10-04 08:58:22 +00:00
stephan
2716cd2528 Minor pstack doc update. 
FossilOrigin-Name: 6b4fb7c4e399ecc948ad41f5fa3139a9293cc822cd7efa2b83dc5fb1660b3ec6
 2022-10-04 01:21:34 +00:00
stephan
2b776ee2d6 Minor cleanups and additions in sqlite3.capi.wasm.pstack. 
FossilOrigin-Name: 97bd670d3859bd3f2e62c15065b238b85ef614f26479e56a14be42c5784aedf4
 2022-10-04 01:11:10 +00:00
stephan
3c272ba380 Add a test/debug mechanism to shut down the OPFS async listener so that it can be inspected (it normally can't be because its tight event-listening loop ties up the thread) and then restarted. 
FossilOrigin-Name: 7d0bcff4e9b899cd25b393b9f0a02c5dcee2e229f0a0fa01719c7dcd7dcbe7c1
 2022-10-04 00:54:00 +00:00
stephan
88838f6b95 Set default page cache size to 16mb in wasm builds. Fix an off-by-one counter in sqlite3_wasm_enum_json(). Minor coding style conformance tweaks. 
FossilOrigin-Name: 72a9e589cc318ec50941739e2edf2c0636284b316c3bf87b71fd363b37619da3
 2022-10-03 23:13:05 +00:00
stephan
faf6a07965 Add a --cachesize flag to the speedtest1-worker-opfs link in index.html because opfs is much faster with that. 
FossilOrigin-Name: 5b8f8e33afc47c9f0c64dbe34643d7d1c37a0a82afb4656e714a86ef54a9ce6f
 2022-10-03 22:51:24 +00:00
stephan
8d9e595513 Minor JS doc updates and typo fixes. 
FossilOrigin-Name: 3cfcc14dfd220536141aeffb902fdc8db1cea055b2a0609b88e092fc3df94688
 2022-10-03 22:38:00 +00:00
stephan
d234902b7c Add configurable cache_size to batch-runner and speedtest1-worker. Add SQL tracing to demo-123.js just for demonstration's sake. 
FossilOrigin-Name: 2ab065133bec19f71a1dc382946fdbd020920b2e38a526f4126a3a1e8df5de87
 2022-10-03 18:07:25 +00:00
stephan
4b074692f4 Omit WAL mode from the wasm build. 
FossilOrigin-Name: 88efe2a62b81e5ee9119f54204e397fda59666a327158c7b7c972e84a716869e
 2022-10-03 18:05:27 +00:00
stephan
57956a9c86 Resolve/remove a TODO. 
FossilOrigin-Name: 2ea35fea3bac2637e935643edfa7913b792e529de3a8980dbb8678d98ed5bcf9
 2022-10-03 13:46:20 +00:00
stephan
4f5bbedb3a Export sqlite3_trace_v2() to wasm and use it to ensure that the new per-VFS post-open SQL support in the DB ctor works. Default opfs vfs to journal_mode=truncate, as it's faster in that mode. Add 't' DB open-mode flag to enable SQL tracing to console.log(). 
FossilOrigin-Name: 508f7f6d63e52f61fae5abe817579a4e130fa7fbd18733d741d521a5bdabb7ce
 2022-10-03 13:03:41 +00:00
stephan
a4c357f94c Reimplement fiddle_reset_db() so that it works with all VFSes. 
FossilOrigin-Name: 18462052903bfeb037dd9af3a1fe1898d141eff5d6f527721858120553de808d
 2022-10-03 11:42:45 +00:00
stephan
5f0b67cb7a Experimentally relinquish the OPFS VFS sync access handle when the db is idle and reacquire it on demand, the goal being to help alleviate cross-tab locking issues. 
FossilOrigin-Name: 2703ac9842335962e488e597168d70b1389b95a6ad39edf70a211b95979b4708
 2022-10-03 11:33:35 +00:00
stephan
33ded6edaf Add journal=MODE to the list of supported URL flags for speedtest1-worker.html. 
FossilOrigin-Name: 5c43e8d2ec23f28fdce63874d9d4c8ccbb4a8f81030b19d65ff5711a7c32697e
 2022-10-03 11:23:47 +00:00
stephan
7ff8da876d OPFS VFS now lazily opens its sync access handle, as a step towards experimenting with relinquishing it during idle times to help avoid cross-tab and page-reload locking issues. 
FossilOrigin-Name: a984e1ba435731413a541f86c50232bc7d6e33aff6ba4cca90f89188e7b82a2c
 2022-10-03 09:21:37 +00:00
stephan
bdfd7ea03a Partial revert of [a82e6faaa642] to remove the 'I' alias for 'j' wasm function signature letter. In hindsight, that change seems premature. 
FossilOrigin-Name: dcd46af9141f4edf816010923941a76d0edd3f18cfe429c52f599ad2a0d52651
 2022-10-03 08:30:22 +00:00
stephan
ef9cd12ec4 Minor JS API tweaks prompted by documenting them. 
FossilOrigin-Name: a82e6faaa642b09d241232c4daa67134d4dfa24bf3ca3725740346ca5269b381
 2022-10-03 08:21:06 +00:00
stephan
510a9d1c64 More cleanups in the UDF argument and result handling, in particular int64. Consolidate some duplicate int64/bigint range checking code. Expose the UDF low-level utilities (arg/result conversion) to client code. Add the sqlite3_context pointer to the JS-side UDF wrappers for API consistency. 
FossilOrigin-Name: 10ab77af952abf09f93f342a9d07a3b133f2c4c0a3588df3390cd3a923cafae4
 2022-10-02 22:50:04 +00:00
stephan
92ede964e1 JS: clean up create_function() wrapper and add support for create_window_function(). Eliminate an extraneous blob copy when a UDF returns a blob. Make use of newfound JS-fu to clean up how sqlite3ApiBootstrap() config is initialized. 
FossilOrigin-Name: d3bad9347c5423fa7f19ae729461636f1043c99a1f01f168efa10bebefb1cdd1
 2022-10-02 20:08:53 +00:00
stephan
9892883e9e js: implement a hand-written wrapper for sqlite3_create_function_v2() which converts, if necessary, JS-function-type args to WASM function wrappers. Replace DB.createFunction() impl with the new one. 
FossilOrigin-Name: 435ab33384017967e46f52b70bee851a85a28808990a0e58dd5288f606b89c9c
 2022-10-02 18:47:39 +00:00
stephan
9fdce595eb Doc typo fixes. 
FossilOrigin-Name: e528675da3971907666c7d2d09763975a105ec585dc5122145f65905d535bed8
 2022-10-02 03:14:38 +00:00
stephan
63e9ec2f9c More fleshing out of sqlite3.capi.wasm.pstack. 
FossilOrigin-Name: eb5726677a727a958df11f1fba078d30c7c0ba2a9bdb158e8641b35b5f971af3
 2022-10-02 03:11:13 +00:00
stephan
6479c5a359 Document the roles of the new (this past week) JS files added to the build process. 
FossilOrigin-Name: 8b3bc7313aff551e5ee0b7aeb927095cf19b9b96abbdd922066c130656b8aa7d
 2022-10-02 01:48:14 +00:00
stephan
193ee11fe1 General internal cleanups in the oo1 API. 
FossilOrigin-Name: f9db664f756f3707afcb5dce87f6d946625848f27ea84337af68de72d4ad6c6b
 2022-10-02 00:09:40 +00:00
stephan
f064a3bbe4 Correct fiddle db export breakage caused by a post-testing API change made in [1fa019c88dea]. 
FossilOrigin-Name: e73cc44ec36c7585ebb914bdeed5b39480fca7c7a8d3c4426bfe769c87b98d17
 2022-10-01 19:21:52 +00:00
stephan
f446af57b5 Pedantic constness tweak. 
FossilOrigin-Name: c8a173cf16d2567bf9b13f52904077a2e7209776c76613c7bff59cd66e24bf11
 2022-10-01 18:55:10 +00:00
stephan
3afad4d432 wasm: correct a memleak caused by a shadowed var in the previous checkin. Add a stack-like allocator, sqlite3.capi.wasm.pstack, as a faster way of managing short-lived pointers (like the one which got shadowed). 
FossilOrigin-Name: 1fa019c88deac6b6e5155b620bdab1d7145846290daafb9adbefcf4f0fe542cf
 2022-10-01 18:47:42 +00:00
stephan
32781427d7 Fiddle: fix makefile dependency issue and duplicate inclusion of post-js.js. Reimplement db export using sqlite3_serialize(). 
FossilOrigin-Name: 29db7de79232c21d19b91bb0fc253114e02e21dd9bf90619453dbe72a4c8bf7f
 2022-10-01 16:01:41 +00:00
stephan
c9099d2d73 Tweak sqlite3-worker1.js to be able to load either sqlite3.js or sqlite3-wasmfs.js, noting that the latter still does not load in a Worker because of an Emscripten loader bug. 
FossilOrigin-Name: 000ef7059bfb54dc4f829b81a8d8c927c0382980218d8a3d60cd2c4d89151c90
 2022-10-01 13:38:27 +00:00
stephan
6167d5cf17 Note two wasm-related potential TODOs. 
FossilOrigin-Name: 5636e82864457d870754ee7125c307dc5d2195197a5c0266579da9f102938b89
 2022-10-01 12:04:49 +00:00
stephan
8c1a4b89af Remove a couple of weird quirks of the Worker1 and Promiser APIs. The Worker1 (now undocumented) unlink capability needs to be reevaluated to work equivalently for all storage backends. 
FossilOrigin-Name: 9f2b331a32cbaadfd20d04c9908171355322c1aa1d9d6df3628f3b2fb4391ec1
 2022-09-30 23:49:43 +00:00
stephan
ae589b69db Tweaks to the Worker1 and Promiser APIs prompted by documenting them. 
FossilOrigin-Name: c68b9aa160e2c1197ae7eb06a634017ac2b281393074afa4582762d5458c6889
 2022-09-30 23:02:11 +00:00
stephan
e67a0f40e4 Add JS wrapper for sqlite3_exec() which knows how to handle a JS callback. Add some console.error() reporting of module-load failures, as they otherwise often get silently swallowed up by the loader's mechanisms. Add 'flexible-string' JS-to-WASM argument converter which performs more X-to-string conversions than the 'string' arg converter does. 
FossilOrigin-Name: 96818aa83f4ccc574f558231249ecbdd39763b4351cf4cf6d33f53774a3ee5e6
 2022-09-30 20:35:37 +00:00
stephan
d18f1bbfe0 Add sqlite3.version object. Add more state to the Worker #1 config-get response, including sqlite3.version. 
FossilOrigin-Name: 711f458d188a0dbe6612069c856ade29323ab426dfa4f80e7b82757ccc474cb8
 2022-09-30 16:49:03 +00:00
stephan
07c0b72253 Add oo1.JsStorageDb.clearStorage/storageSize() methods, copies of capi.sqlite3_web_kvvfs_clear/size(). 
FossilOrigin-Name: 1e09efe7fa15b8908f8b8353164a8361de778e27ea6c0b11c402bf4e1c56333d
 2022-09-30 15:46:08 +00:00
stephan
359d62395e batch-runner.js: force WebSQL batches to not abort for a failed statement (necessary for apples-to-apples-ish benchmark comparisons). 
FossilOrigin-Name: 60f0c5cb04d2baf8431e523434d8753e39377f4b2c6bad225c2e5487a4be419b
 2022-09-30 15:24:58 +00:00
stephan
dc6ae60226 Expose sqlite3_msize() to wasm. 
FossilOrigin-Name: aa6ad34f3b693b554be58ee2895f79397162c5b8877c709653c8508fc625e8f4
 2022-09-30 12:10:36 +00:00
stephan
f6c686c9f4 Add oo1.JsStorageDb() as a convenience wrapper for oo1.DB(...,'kvvfs'). Minor doc cleanups. 
FossilOrigin-Name: 8a7998709f859a562cf6829485cb9921f8823af0efabe003741348ab1169fb89
 2022-09-30 11:01:44 +00:00
stephan
53d4e01d06 wasm: expose sqlite3_de/serialize(), sqlite3_malloc/free() and friends, noting that the former explicitly lies on use of the latter for memory management so is not generically safe for use in wasm. 
FossilOrigin-Name: fbc0edb5d31aa0dea92460e853f15f08c642451a7878994116b530cf172325cc
 2022-09-30 10:55:28 +00:00
stephan
f71c954cbc Add JS infrastructure to ostensibly allow us to customize the wasm imports, which will hypothetically allow us to eliminate the dependency on EM_JS(), but the corresponding Emscripten glue-level feature currently breaks fatally with WASMFS builds so it's disabled. 
FossilOrigin-Name: 88d9253b0db5494bf1c9b6d24f22524eeec856b89e64a66ffb30d945f0df21d3
 2022-09-29 22:08:22 +00:00
stephan
b0ccf50cbd Ensure that the wasm builds always, even when passing -Ox flags from the CLI, use -g3 to avoid problems caused by symbol minification with optimization levels -O2 and higher. 
FossilOrigin-Name: 27d24d5d270e272af07d96cf07bdbe79ba07a9c69ba3c70beb76ebf4f90fc97a
 2022-09-29 17:35:25 +00:00
stephan
eb97743cc7 Finish eliminating explicit Emscripten module dependencies in test code and fiddle. The only remnant in public code is the Emscripten-generated module load/init interface. 
FossilOrigin-Name: 7be78dd4efc410f13ff1ceda1fad82b309cc24de2d5106c8bd6b2adeaa13b106
 2022-09-29 16:54:23 +00:00
stephan
b94a98607a Rework the Emscripten-emitted module loader/init function such that it passes on the sqlite3 module, instead of the Emscripten module, to the first then() of sqlite3InitModule()'s returned Promise. This eliminates any need to mention the Emscripten module object in client-side code unless they want to configure it in advance for loading-status reports. 
FossilOrigin-Name: 0dbaa0e2b5abf5c23e2039ec90a3055ebb3c063aaf4e556c42546defe6fbb86d
 2022-09-29 13:17:50 +00:00
drh
31474ce57e Fix misuse of the sqlite3_set_auxdata() interface in the ICU extension. 
FossilOrigin-Name: f25cf63471cbed1edb27591e57fead62550d4046dbdcb61312288f0f6f24c646
 2022-09-28 19:14:01 +00:00
stephan
4b884bb4c7 Wasm: expose sqlite3_exec() and use it to simplify the db-reset logic in batch-runner.js a bit. 
FossilOrigin-Name: 2e2821f782511b9d2274a89a5a922582aba18c7e9dc7ce01080e713942a56d7d
 2022-09-28 18:10:50 +00:00
stephan
cb22bd80d2 More work on batch-runner.html/js to facilitate speed comparisons between various VFSes and WebSQL. 
FossilOrigin-Name: 3bd1bc240676e56da87fc49f3c67a1edc4fafcf2a2416298d19ae4f80b676a72
 2022-09-28 17:52:52 +00:00
stephan
5ad3631915 Correct duplicate copies of sqlite3-api.js being embedded in the wasmfs-based builds. 
FossilOrigin-Name: bbfcfba260f39a9c91e82d87e06b1c2cb297c03498b4530aa3e7e01ef9916012
 2022-09-28 13:01:49 +00:00
stephan
d98011852d Work around broken -Os wasm builds by adding the -g3 flag. Unrelated documentation tweaks. 
FossilOrigin-Name: f5d6bf8616341037fa3e229edf820d19acef3e0a6207a652b2b143de0a493214
 2022-09-28 07:53:47 +00:00
stephan
489a8a9d3c Minor doc correction. 
FossilOrigin-Name: 093f6e4b36db294e8e47df2fc75a4bc4fc101e2b6ff0201c912ccd1dcf394479
 2022-09-27 17:03:01 +00:00
stephan
6559e0ab15 Comment the -O2 wasm builds as being broken (mysteriously missing JS bits). 
FossilOrigin-Name: f61475ef8244fd1f6553eca9cc2317e5de2cc04443e24afb7b91ebd3fc1bd402
 2022-09-27 14:31:34 +00:00
stephan
5b9973d898 WASM API renaming. Reworked JS API bootstrap's async post-init into a generic mechanism, no longer OPFS-specific. 
FossilOrigin-Name: c42a8cb090cad1108dfd6be574202d744c59e053b505bc4c17252dc6b65d26bf
 2022-09-27 13:40:12 +00:00
stephan
3d64548491 wasm/js: rename /persistent to /opfs to account for potential future persistent storage options. Minor flag-handling cleanups in the speedtest1 pages. Minor API tweaks in oo1. 
FossilOrigin-Name: 4dc972a3656b2a9ec915bfb3f653136560c753ce4024c3f0d0d0c28f66db7a0a
 2022-09-27 09:17:37 +00:00
drh
6319a8003c Make the legacy FTS3 code more robust against integer overflow during 
memory allocation.

FossilOrigin-Name: 5517bc50988b5339c2fd071b29de1b5ca03037b0b635c3b112cf7108fab54d5f
 2022-09-27 01:53:05 +00:00
dan
7e33f8f43b Modify implementation of recovery to break recovery of the lost_and_found table into multiple steps. 
FossilOrigin-Name: adedfd040bb2d1c2dcda0b91c6073f12953e45d9b860a8f217d990e861d66dfb
 2022-09-26 20:51:16 +00:00
stephan
278d3faf1f Fiddle: replace db export routine with a C-side one which works for both Emscripten FS-hosted and OPFS-hosted db files. Minor code-adjacent cleanups. 
FossilOrigin-Name: 3579a8d6f1f6cd3cd8aad9949536870c5fe7bae8c1778f700dd85d763e266b94
 2022-09-26 13:55:10 +00:00
stephan
1f095d482d Get fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS worker to enable creation of certain utility functions without delegating to the async worker. Add sqlite3.capi.sqlite3_wasm_rc_str() to map integer result codes back to their SQLITE_xxx counterparts. Minor doc touchups. 
FossilOrigin-Name: 9b2244e1c8a40efe6547094a1b57acc8f2173145a8731abb0e36268ce0a8ef41
 2022-09-26 11:38:58 +00:00
stephan
6a643e4bcd wasm: change StructBinder signature for sqlite3_file::pMethods from 'P' to 'p' to eliminate an unnecessary and inconsistent level of magic. 
FossilOrigin-Name: 85f2e877e53899860af4dc6630044b471a9c7c82faba1f4e1e60ae991460b943
 2022-09-26 11:34:31 +00:00
dan
8d8fc58413 Add the SQLITE_RECOVER_SLOWINDEXES option, for specifying that indexes should be created and populated along with tables, instead of separately at the end of the recovery operation. 
FossilOrigin-Name: ad9dba9d1eae786575c7f31e34b342b6f5b26e719bbe27b61609cad8cfd0a505
 2022-09-24 19:54:49 +00:00
dan
351563efe1 Add sqlite3_recover_step() to header file sqlite3recover.h. Update the API docs in this file. 
FossilOrigin-Name: 47f416153035d6bf0ae27587583d2957c87c02caecbd4e7f363bcb2bc27cf159
 2022-09-24 19:17:20 +00:00
dan
aee4049128 Update things to use sqlite3_recover_step() internally. 
FossilOrigin-Name: f4b15aad3005237b7ac507eed2b9b07e0f5c9407ab28f2656a21c9845f13d35f
 2022-09-24 18:05:52 +00:00
stephan
395012e58e Fiddle: move, rather than copy, data buffers between the threads. Fix the case of an uploaded db failing to install because its filename is the same as the opened db. 
FossilOrigin-Name: bcec4f964a7b02f59be05286ff715bac654a32b19f05a743e402f4cdb207cab8
 2022-09-24 11:32:00 +00:00
stephan
ef11fb915d Resolve "already configured" warnings from shell's main() when starting it up in fiddle mode. 
FossilOrigin-Name: 114ef3552af977b272a0baddeb9a2326484b60acfc75284e43c55530f86b413f
 2022-09-24 10:15:08 +00:00
stephan
56fae744d4 Reworked out the OPFS async proxy metrics are fetched so that they play more nicely with the tight event-polling loop. 
FossilOrigin-Name: ef503ced5c2ca842be9aea9ef13719a378ed3020e884032db09afee1b8eba0a1
 2022-09-24 10:12:19 +00:00
stephan
60d9aa7c59 Refactoring towards getting fiddle to support OPFS as a first-class citizen. Certain operations, e.g. import, export, and unlink, are not OPFS-aware. 
FossilOrigin-Name: 1b923ed6438d7fef4508936e0c4bc026a368721698b1539961e3fb3140a185cb
 2022-09-24 07:36:45 +00:00
dan
9bbd91b39d Internal changes to the recover extension towards a step() style interface. 
FossilOrigin-Name: 73926d5c8cd1ecece134b5a73b44ee1dfca74dc200606e3f009b06cdecf8cee9
 2022-09-23 21:01:10 +00:00
dan
80b1f6fcde Fix various compiler warnings in new code on this branch. 
FossilOrigin-Name: ae49e9efde3012158061def6e0a8d993abbc5933514a21f84bc10f700f61b5e2
 2022-09-23 11:40:43 +00:00
stephan
f75c0c7036 Reformulate some JS to work around a buggy/broken code transformation in one of the Emscripten-driven code optimizers. 
FossilOrigin-Name: e1249369d5ec1c582c280b1f578b35d53637fdf1cbd97c16d5ed95b136b83e56
 2022-09-21 20:24:12 +00:00
stephan
6110a5d0ab Put pieces in place for fiddle to support opfs, but more cleanup is required in the sqlite3.js/fiddle connection. bigIntEnabled now defaults to whether the Emscripten's module has bigint enabled. Add wasm-sensible defaults for several SQLITE_ENABLE/OMIT flags in sqlite3-wasm.c. 
FossilOrigin-Name: 7c7fb7535e86b3960eea7f29ab7e6d5197c166b4ee64ad4a9bc0749f2869badc
 2022-09-21 19:51:25 +00:00
stephan
e8afca3f16 Correct mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups. 
FossilOrigin-Name: 86e2b55ec9483fa5add51a479c6509d73461f1ac6fca5d49e057b1c66f4314d2
 2022-09-21 14:02:47 +00:00
stephan
72ab400d4d Doc cleanups and additions. Add a way for the OPFS async runner to propagate exception text to the calling thread. 
FossilOrigin-Name: 5c5e80652825cf883e6c17809cb98f2bf17d5feac2d263f6f492479154730dab
 2022-09-21 12:27:35 +00:00
stephan
171b168b3e Added some structure to the wasm demo link list. 
FossilOrigin-Name: 777077c4c2249e1ec78390d4f65aaf281c1fbefcef4bcc7609199e995645ceb6
 2022-09-21 12:25:40 +00:00
stephan
ebe1423098 Add a link to fiddle.html in the wasm test app index.html. 
FossilOrigin-Name: 96c734c07acfbea153d9aaf293a28a2d49d78de19fec4fb90c4c08fb1fd0ddbd
 2022-09-21 08:41:40 +00:00
stephan
4bc2f6b415 Move fiddle build rules into the wasm-centric build files. Add rule to push wasm bits to the wasm test server. 
FossilOrigin-Name: 113f8204dc4ac849d5632d3de1680b6e0da871e107ef484c8d7273799bee3d88
 2022-09-21 08:39:03 +00:00
stephan
4cffb644c7 Correct a too-strict is-opfs-available check. 
FossilOrigin-Name: 1b5f1b4a6c1457f98c258459e23e321fc59793de298fecb84031b87f02156cd5
 2022-09-20 16:20:35 +00:00
stephan
fa5aac7475 Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks. 
FossilOrigin-Name: e3d36dcdd37e59f17a07d3611d08744eb86f439fab82a648490dd608bcaa3185
 2022-09-20 16:10:39 +00:00
stephan
89071030db Export the sqlite3_uri_...() family of functions to wasm. 
FossilOrigin-Name: 72bebc848fce5c3b4766017d016ccb2360de2bd0cb3e47e710c80dbcb6b8b707
 2022-09-20 14:52:26 +00:00
stephan
0e0aafc698 Remove an obsolete file. 
FossilOrigin-Name: 1e43855498b4329f733bd6e0731841dc884e885de7d2d338d402d05d54309427
 2022-09-20 14:21:58 +00:00
stephan
b8c8d4e4a0 Speed up de/serialization of func args and return values in the OPFS VFS proxy. 
FossilOrigin-Name: 5bf235bbe035e4ace7a54851e190742528af6b4266328a1b8bbb9fb3dd7f2118
 2022-09-20 13:25:39 +00:00
stephan
c2ccd67689 OPFS proxy: remove one sanity-checking-only level of proxy function to shave off a few microseconds. 
FossilOrigin-Name: b534831f3efb8910a17e29956e3e87cc80055ea66e15dbef992b6a556ff042f8
 2022-09-20 10:47:36 +00:00
stephan
c9e2602e8f Remove extraneous result-code SAB slots. Doc additions. 
FossilOrigin-Name: 88de20712bbe3e11ea74af95edc34e9ab9f346f9aa0a30931e5a9e1f96ce57b0
 2022-09-20 10:11:52 +00:00
stephan
5e8bb0aa13 An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS. 
FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a
 2022-09-20 08:27:57 +00:00
stephan
138647a552 OPFS VFS: further internal refactoring towards experimenting with a new comms model. 
FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1
 2022-09-20 03:31:02 +00:00
stephan
c4b87be3e8 OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model. 
FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d
 2022-09-20 01:28:47 +00:00
stephan
aec046a264 Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect. 
FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96
 2022-09-19 18:22:29 +00:00
stephan
f815011a08 Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th. 
FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51
 2022-09-19 17:09:09 +00:00
stephan
61418d5a67 Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number. 
FossilOrigin-Name: 976053925013bf1975f5f9222e28ba648af28e305bb6bdae600eb24d0e136bec
 2022-09-19 14:56:13 +00:00
stephan
0e0687ccfc Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS). 
FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48
 2022-09-19 13:44:23 +00:00
stephan
b5ae85eca2 Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl. 
FossilOrigin-Name: ec09f32f7ae2249aaf27388ad2062982afa8bbbb5f88d236d6d9068bf33ad93d
 2022-09-19 13:16:35 +00:00
stephan
ba0b157583 Minor doc improvements in demo-123.js. 
FossilOrigin-Name: 9370ce94780e974da03d5c74cf9a5983eed333e7b3bc54e605c5f30b3fa99287
 2022-09-19 10:13:01 +00:00
stephan
509f40526e Generic cleanups in the OPFS VFS proxies. 
FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7
 2022-09-19 09:58:01 +00:00
stephan
862281fc47 Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ. 
FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d
 2022-09-19 09:25:25 +00:00
stephan
8766fd20d1 Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed). 
FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5
 2022-09-19 05:19:04 +00:00
stephan
ac51eb7754 Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 
FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764
 2022-09-19 03:57:31 +00:00
stephan
ed85333977 Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there. 
FossilOrigin-Name: 9a9eeebc2c27b734041089131b4952d7c0440df48ef32f355641aca61d4b30a0
 2022-09-19 00:40:53 +00:00
stephan
f386012069 Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 
FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8
 2022-09-18 17:32:35 +00:00
stephan
0db3089576 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS. 
FossilOrigin-Name: b2abf60dbfa6648f671a3932cb65feb28d05a0d5b7f792351d14f9c13d9798c5
 2022-09-18 03:05:55 +00:00
stephan
c5313afea7 Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. 
FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3
 2022-09-18 02:35:30 +00:00
stephan
3e2823cbb6 Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code. 
FossilOrigin-Name: a0e93ed20b2463606a63b03ce8ca41ec1fb22886db5c5c898ace86ba24636f70
 2022-09-18 00:16:12 +00:00
stephan
5f4ad92681 Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them. 
FossilOrigin-Name: cd06cc670029763955cf60ffcf944b36d41cb005b859d9b9fd0eea1b6741d0e9
 2022-09-17 23:47:20 +00:00
stephan
8200a6d8d7 Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file. 
FossilOrigin-Name: c342b5d745f301104c59851c753287ebbbe95a69a56cb522d376d0f3e352c30f
 2022-09-17 23:29:27 +00:00
stephan
e6f8a09532 Generic cleanups in the OPFS sync/async proxy. 
FossilOrigin-Name: f36bddbe54c3acbfaa958042e4d24724f130bdca551401033f9bc63f3da73492
 2022-09-17 21:13:26 +00:00
stephan
0731554629 Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested. 
FossilOrigin-Name: 44db9132145b3072488ea91db53f6c06be74544beccad5fd07efd22c0f03dc04
 2022-09-17 20:50:12 +00:00
stephan
132a87baa3 Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API. 
FossilOrigin-Name: 38da059b472415da52f57de7332fbeb8a91e3add1f4be3ff9c1924b52672f77c
 2022-09-17 15:08:22 +00:00
stephan
ce260f4e43 Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit. 
FossilOrigin-Name: afb79050e635f3c698e51f06c346cbf23b096cfda7d0f1d8e68514ea0c25b7b7
 2022-09-16 20:16:50 +00:00
stephan
81439a07f0 When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters. 
FossilOrigin-Name: 52d1b185b9f6cee1eb3dec436f47e0f52e4621a127abfad8c27f92fd78147889
 2022-09-16 12:49:45 +00:00
stephan
4c72171b60 Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code. 
FossilOrigin-Name: 13899bb98c80525276d2484598b94e4206358f243f06d45c02700024f7e226fd
 2022-09-16 11:45:09 +00:00
stephan
57db2174d5 Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. 
FossilOrigin-Name: d8df25920a047d5cf2093cc6233128c5e6057a9104d0c4397e643645bd646fe1
 2022-09-16 02:30:49 +00:00
stephan
58bcfe075d Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args. 
FossilOrigin-Name: ad0677e8abcc077636d1cf1d8485be4943506382581edf832e6b8a2021560040
 2022-09-16 02:28:03 +00:00
stephan
0144a8482a Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7. 
FossilOrigin-Name: 13839759f8f45e4eb0aa6a5052801f3964d5b2dc2e427e5e91ee6692176381eb
 2022-09-16 01:05:19 +00:00
stephan
28ef9bddb3 More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 
FossilOrigin-Name: 00ee49a3a2c148480f614e49a0ee5b35a255758c0a53693f0b464b31e7a4045b
 2022-09-15 06:42:41 +00:00
dan
73390f09c2 Fix windows builds of recover extension. 
FossilOrigin-Name: abcbb6abfe08fc590123f0aa1bd72cfdecc75908f078f1348dc0e957de98bf52
 2022-09-15 03:54:52 +00:00
stephan
dae7518ae4 Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API. 
FossilOrigin-Name: 4cbbd370186f84e440f496e7f11c641d7dc1723acc46e4a31483392e0eb046e9
 2022-09-15 03:16:49 +00:00
stephan
0117693f62 Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 
FossilOrigin-Name: 08476f3c218d45846e7496bdae0b06e2122466111fdf2aa2aabb1805b15ef982
 2022-09-15 03:09:00 +00:00
stephan
409505c743 Clean up some JS documentation. 
FossilOrigin-Name: 925fdbfc6c1f9f06951346bc0d4fb81ffb5797d5f1123728931973f6cb1a0efa
 2022-09-15 02:27:48 +00:00
dan
6fa0a11c7b Include recovery extension in configure/make builds of testfixture. 
FossilOrigin-Name: d2f4652144c4a54012a2176c178c5ade3463bd017515f472cfa8dafdde192df3
 2022-09-14 20:45:46 +00:00
dan
3c24342841 Extra tests and fixes. 
FossilOrigin-Name: deed5336931b23bb507c064d08e9899b33f04e4f7eee03beb8d7147eb4caa030
 2022-09-14 18:57:46 +00:00
dan
b8eaf9a10d Fix recovery of utf-16 databases. 
FossilOrigin-Name: 5b05be0861f35804270fbd184ad4b89c23e98cc2fbd56b9e4fe6197daef5fe49
 2022-09-14 16:37:59 +00:00
dan
322967df59 Add OOM tests for the recovery extension. 
FossilOrigin-Name: 9b6b4c7162439a889144edb561356afc66436db921a867c20871f0c556716502
 2022-09-13 20:40:57 +00:00
dan
a3a6d63b40 Fix a broken error handling case in the recovery extension. 
FossilOrigin-Name: 60089547e1fc77ecc02f207ebf75ee3160e5ff25f41d12e02e170fd7fde66602
 2022-09-13 19:47:01 +00:00
stephan
5b91500796 Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. 
FossilOrigin-Name: 0d78961870ee9f22f1ba16d423377d28dcc36e04b1e31ffd57f3e2fd51f8f0f2
 2022-09-13 19:27:03 +00:00
dan
7985e05a97 Add tests. Deal issues surrounding aborting recovery from within the SQL callback, and avoiding the pending-byte page. 
FossilOrigin-Name: 4e97dd31f5240d9231167ae172a5116426c42177a1ed3c5422b9d51b762d5a87
 2022-09-13 18:08:24 +00:00
stephan
23d39ce54d Get testing1.js working with a kvvfs build. 
FossilOrigin-Name: 333a45725d1708e0fefa559c33ce1c7eeb425cdb04f594b1f2b48166c1478c79
 2022-09-12 22:27:00 +00:00
stephan
0b769f5b4a Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on). 
FossilOrigin-Name: 8b1608a9a37131121cb3a0cb1e8dbad5e39dc1fc2a341d056f09537f179b2e7a
 2022-09-12 20:18:28 +00:00
dan
5ca0b38605 Fix some problems with handling corrupt database in the recovery extension. 
FossilOrigin-Name: ed318be8241981ef96334ba13d3201a717cc812a59aed64e3dc67f7e7e71854b
 2022-09-12 20:02:33 +00:00
dan
aede9b5bd3 More tests for the recover module. 
FossilOrigin-Name: 37fb093b95c6b7d7ad07a275697df73b69f9fb5c5549aea8544b26e38f24833f
 2022-09-12 19:23:50 +00:00
stephan
a002cc174f Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console. 
FossilOrigin-Name: d845c6c22bd5d3fffc66e0566df346d690dd8bd1fc1688e312161b1a1edcfd79
 2022-09-12 17:59:04 +00:00
stephan
5360f5fcff Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo. 
FossilOrigin-Name: a7d8b26acd3c1ae344369e4d70804c0cab45272c0983cfd32d616a0a7b28acb9
 2022-09-12 16:09:50 +00:00
stephan
2811759d5e Resolve missing kvvfs JS test: sqlite3_initialize() must be called by the client in this build. Add basic sanity test of kvvfs using sessionStorage(). 
FossilOrigin-Name: 250a935aeb94d3fadec0d3fe22de85de4e658e2fdb3be3aa9a8bbc8f7b7d8414
 2022-09-12 13:05:54 +00:00
stephan
ad5125a624 Initial build of kvvfs in wasm. It loads but cannot find the VFS for as-yet-unknown reasons (sqlite3 shell works fine), and most APIs throw "null function or function signature mismatch" from deep within wasm, presumably as a side effect of the "missing" VFS. 
FossilOrigin-Name: 1a2f24a0bdfc6eaae478916b8f4f9c6b63ead9964534fc2951fb4e995ffe61f1
 2022-09-12 12:39:28 +00:00
stephan
73079dba00 Minor cleanups and documentation in the wasm pieces. 
FossilOrigin-Name: 4e6ce329872eb733ba2f7f7879747c52761ae97790fd8ed169a25a79854cc3d9
 2022-09-11 16:59:40 +00:00
stephan
2a91126d76 Correct the result code from kvstorageRead() for the OOM case to be a negative value. 
FossilOrigin-Name: cdbf09fa1b0c93aeb3222a157de33a4688ae629c2b829ffff0f1f62364c5ae1c
 2022-09-11 05:44:15 +00:00
stephan
ade4f4d645 Minor cleanups and docs for the EM_JS() impls of the kvvfs read/write/delete ops. 
FossilOrigin-Name: 53b7572e441be4b4b29b4228d0f95b24457d7faaf6c0b568ed5c970a55e21ffb
 2022-09-11 05:38:39 +00:00
stephan
e99f742a32 Add EM_JS() impl for kvstorageRead(). 
FossilOrigin-Name: 06610314fcf644f323c2f7ae11d7f4349b195e66d0ebbee590438dd99d97eb96
 2022-09-11 04:49:29 +00:00
stephan
d15822965b Initial prototype impls of write/delete-key ops for the pending kvvfs which use sessionStorage or localStorage for db page storage. read-key op is pending. 
FossilOrigin-Name: a4bd96f5348e607d0fcb627b751f9d86a188173a4b3bfb2c95f92913a78bd31e
 2022-09-11 02:43:08 +00:00
dan
65660916dc Rework recover extension code for readability. 
FossilOrigin-Name: 1a2540960e40e3c8c622448fd3862e249bd463c29ae4ce5e39942e942533f60a
 2022-09-10 20:01:49 +00:00
drh
7585f49a0a Move the vfskv.c extension to src/os_kv.c and make it part of the amalgamation, 
activated if and only if SQLITE_OS_KV is true.

FossilOrigin-Name: f6632e69c2ec1a7ddc4e51f3567e3bc082ee94a6dd51fdafdc0c3bf386a32d4c
 2022-09-10 18:20:59 +00:00
dan
bc2e7fc228 Further fixes to comments in sqlite3recover.h. Also rework some data structures in sqlite3recover.c. 
FossilOrigin-Name: 599d1f8ec2f9e24924a6f9e66c85664360c7b95531b07a4efe1dd8c096b3fc99
 2022-09-09 20:44:56 +00:00
dan
7920162093 Handle recovery of virtual tables by recovering each shadow table individually, then writing the CREATE VIRTUAL TABLE statement directly into the sqlite_schema table. 
FossilOrigin-Name: 5f2d5ccd56c06c3468377126acfd4be39b79b05bb6fb09b674b2e185df143aa3
 2022-09-09 16:25:19 +00:00
drh
25f62508f5 Database must be named either "local" or "session". Each has a distinct 
namespace.

FossilOrigin-Name: c8e41279294ea7c2f041c7d4cbbd85fce47d55f2f56180ca837316f7dfd75237
 2022-09-09 16:16:33 +00:00
drh
f7377d5763 Separate sqlite3_io_methods objects for database files and journals. 
FossilOrigin-Name: 74fbf6c2a97729e481d5562344e960d0c237aeb1f07240262d32a24fb1f4b5a3
 2022-09-09 14:22:41 +00:00
drh
126976c9f8 Simple reading and writing now works. 
FossilOrigin-Name: 00845ac9ef2616e90f5f6a20da3dd040fa8bdcfe73f72fe9c06039561150a82d
 2022-09-09 11:41:54 +00:00