stephan
|
04184761de
|
Internal restructuring of the OPFS sqlite3_vfs in order to facilitate certain experimentation and improve error reporting/hints if it cannot be activated. Deprecate the name sqlite3.opfs.OpfsDb, preferring sqlite3.oo1.OpfsDb for consistency with JsStorageDb and any future DB subclasses.
FossilOrigin-Name: 0c5c51f4fb04a4b90c50ec9704cfea9a3fb7d7d0ee55c1b0d4476129188217a6
|
2022-11-29 05:25:08 +00:00 |
|
stephan
|
e6b0154138
|
Add an explicit warning about the current API-instability of the sqlite3.opfs namespace, which may need to be eliminated based on re-thinking of how the OPFS sqlite3_vfs is registered. Comment changes only - no code.
FossilOrigin-Name: 0cb2fd14179397051a25d066256a553fc198656d5668c7010c016f2b8f495bf4
|
2022-11-29 02:23:12 +00:00 |
|
stephan
|
df5d06d03e
|
More work on the OPFS concurrency testing app.
FossilOrigin-Name: c0458caca3508d5d252f9b5198bda4f51a5c1874540f014b17e409f2daab1706
|
2022-11-24 17:53:09 +00:00 |
|
stephan
|
e79cb67c35
|
Add an experimental OPFS VFS-specific URI flag, opfs-unlock-asap, which tells the VFS to release implicit locks ASAP. This permits higher concurrency but hurts performance considerably. This may or may not be obsoleted by other concurrency-related experimentation.
FossilOrigin-Name: d23c917013485ec2793125221f3936b05c39d6eca941629fb819b6b4aa714520
|
2022-11-23 19:03:22 +00:00 |
|
stephan
|
ad1285c5c0
|
Initial infrastructure for adding a mode to the OPFS VFS which causes implicit locks to be released ASAP, which increases concurrency at the cost of performance.
FossilOrigin-Name: c5b7a9715a13b696ab3ee965aa0a310f59b65f07cecd72faa2e3504bfd8eb632
|
2022-11-23 16:39:07 +00:00 |
|
stephan
|
36d5554c9a
|
Resolve missing SQLITE_LOCKED result code which triggered a new (since last checkin) exception in the OPFS VFS. Improve output of the OPFS contention tester app.
FossilOrigin-Name: 2debbbca33bd4170a1dc4dbb5eb3e68523e51d289b06c551e5560ac4e32e433b
|
2022-11-21 04:12:38 +00:00 |
|
stephan
|
27c4cd183d
|
Add test app for experimenting with multi-worker OPFS concurrency. Tweak OPFS VFS to significantly improve the otherwise "unfortunate" concurrency situation.
FossilOrigin-Name: 96f76e7616f8157a342b9e1c42f7b1feab200d182268871a2b25f67d4ee2564c
|
2022-11-21 03:50:52 +00:00 |
|
stephan
|
0a94ef8b52
|
Minor cleanups in the ESM-related preprocessor filtering.
FossilOrigin-Name: 205884a273128bb666b496b659b4fa9f031ebdbbc1aa704fdeb4b7e015740098
|
2022-11-20 01:15:22 +00:00 |
|
stephan
|
27a67968af
|
Add build of sqlite3.mjs (ES6 module), add a test app for it, and include it in the dist build.
FossilOrigin-Name: 2e783670e10b59e67c14b0db7f4803b41790cc7730de221d54fa2d4483cfba33
|
2022-11-19 05:26:45 +00:00 |
|
stephan
|
ee026c5479
|
Replace use of cpp with the fit-to-purpose c-pp to avoid cpp's C-centric/JS-unfriendly quirks.
FossilOrigin-Name: 49d70f071e918d5d095c807575bb7ce2b287a123261e789e938521b3b409429a
|
2022-11-18 02:29:59 +00:00 |
|
stephan
|
c7c15d1b83
|
Merge trunk into js-cpp branch.
FossilOrigin-Name: e047b33d1fb7d6a32e967f03f9952249cd2da4d21dc301fe92bd7baa0da5d6a9
|
2022-11-17 15:21:49 +00:00 |
|
stephan
|
da2641597a
|
Rework automatically acquired OPFS locks to be released during idle time. This eliminates the performance hit reported in [46304ba057707c].
FossilOrigin-Name: a7fe91afca473fe55c983bc81d214df4ef3699863c7423fa4b6b9cde23d6a3b4
|
2022-11-10 13:14:30 +00:00 |
|
stephan
|
b0ab21dee4
|
Experimentally use clang's C preprocessor to filter #ifdef's out of the generated sqlite3-api.js, the goal being to see if we can feasibly use cpp to include ES6 module-specific code in the main code base and conditionally filter it out.
FossilOrigin-Name: 718a6d371e61359d73c8f80afdb248e3d9b4d8df4c4e5c122ac884344e31035b
|
2022-11-03 22:14:47 +00:00 |
|
stephan
|
f45c33701d
|
Add sqlite3_wasm_vfs_create_file() to replace Emscripten's FS.createDataFile() in a (mostly) VFS-agnostic way. Add a test for worker1's export (to bytearray) support. Re-add worker1 open-from-bytearray using sqlite3_wasm_vfs_create_file() but it's untested (requires a new interactive test app or maybe reconsideration).
FossilOrigin-Name: b35e1225c91a3cadc0d25af1e4e790237256d194990faa13190e343ed03e11c5
|
2022-11-02 11:53:31 +00:00 |
|
stephan
|
49048b148e
|
Significant cleanups and expansion of the sqlite3.opfs utilities. Add oo1.DB.dbVfsName(). Add VFS name to worker1:open's arguments and result.
FossilOrigin-Name: 86a341d7e061f946b39e8647ddd4743013b851b33ae9e6e755d8dbc53fba5286
|
2022-11-01 07:49:49 +00:00 |
|
stephan
|
43b442a626
|
Internal tweaks to the OPFS VFS result codes.
FossilOrigin-Name: 32e1a2d2eb8738ae1635e413829f6cf7f64c63d2a86b72940a573de89948e529
|
2022-10-31 11:53:34 +00:00 |
|
stephan
|
8948fbee29
|
Move the sqlite3.capi.wasm namespace to sqlite3.wasm. This causes a tiny bit of naming confusion with the sqlite3.wasm *file*, but seems to make more sense than having it as a sub-namespace of capi.
FossilOrigin-Name: 3f16eb18d6186ca972fca952ccac18649e7a905213f589e53c0c9333e695448d
|
2022-10-29 07:54:10 +00:00 |
|
stephan
|
e681b651eb
|
Minor sqlite3.oo1 API reorg prompted by documenting.
FossilOrigin-Name: 55237924ca363c5a3e0f8c82620eb6626a98311615866197375a86876b9da3a1
|
2022-10-28 10:36:18 +00:00 |
|
stephan
|
f861b36bf4
|
Minor cleanups and doc improvements in the OPFS sqlite3_vfs proxy.
FossilOrigin-Name: 48645f7bcacf81c4149f26d20ee1752fbe93a02f96b85bd7e28bfa49322137e5
|
2022-10-25 08:06:17 +00:00 |
|
stephan
|
cd0df83c15
|
Apply considerable acrobatics to get the JS/WASM deliverables building to and loadable from a directory other than the one which contains the app-level code. Requires an only-slightly-leaky abstraction of passing a URL argument when loading sqlite3.js but provides much greater flexibility in where the JS/WASM files are located.
FossilOrigin-Name: 6d468dab9eb84d4548f68014959f02fe4f66455472ff24fe729382bb2972e3d1
|
2022-10-19 04:44:58 +00:00 |
|
stephan
|
ff891b4e1c
|
Minor cleaups in the post-init async phase of sqlite3 module initialization.
FossilOrigin-Name: 2e024a6b533524b7732cea8aa27b16363e17b4c32731ddf31085e91fc728c1af
|
2022-10-09 15:12:37 +00:00 |
|
stephan
|
9a55773b2f
|
Replace time-based auto-unlock of opfs sync handles with lock acquisition/release via sqlite3_io_methods::xLock/xUnlock().
FossilOrigin-Name: 2625b7cfe1640c1d7e779ec1f37db970541598c0dc3e22e5eecf3c772d95ad40
|
2022-10-04 17:06:51 +00:00 |
|
stephan
|
ed3182f233
|
Tweaks to the opfs async wait/relinquish times.
FossilOrigin-Name: 35f33c23e5849de1c43c4499ee0a7fa11d26ae34949c1e820c3fa8e8873f9c2b
|
2022-10-04 11:14:23 +00:00 |
|
stephan
|
c7fb48d4b6
|
OPFS async proxy: add a wait-and-retry policy to the get-sync-handle step to help account for cross-tab locking.
FossilOrigin-Name: 45c48c63d311052105d102189208495b2b53fa3c4174884ecf63b757aa4016e5
|
2022-10-04 09:12:05 +00:00 |
|
stephan
|
3c272ba380
|
Add a test/debug mechanism to shut down the OPFS async listener so that it can be inspected (it normally can't be because its tight event-listening loop ties up the thread) and then restarted.
FossilOrigin-Name: 7d0bcff4e9b899cd25b393b9f0a02c5dcee2e229f0a0fa01719c7dcd7dcbe7c1
|
2022-10-04 00:54:00 +00:00 |
|
stephan
|
4f5bbedb3a
|
Export sqlite3_trace_v2() to wasm and use it to ensure that the new per-VFS post-open SQL support in the DB ctor works. Default opfs vfs to journal_mode=truncate, as it's faster in that mode. Add 't' DB open-mode flag to enable SQL tracing to console.log().
FossilOrigin-Name: 508f7f6d63e52f61fae5abe817579a4e130fa7fbd18733d741d521a5bdabb7ce
|
2022-10-03 13:03:41 +00:00 |
|
stephan
|
f6c686c9f4
|
Add oo1.JsStorageDb() as a convenience wrapper for oo1.DB(...,'kvvfs'). Minor doc cleanups.
FossilOrigin-Name: 8a7998709f859a562cf6829485cb9921f8823af0efabe003741348ab1169fb89
|
2022-09-30 11:01:44 +00:00 |
|
stephan
|
6559e0ab15
|
Comment the -O2 wasm builds as being broken (mysteriously missing JS bits).
FossilOrigin-Name: f61475ef8244fd1f6553eca9cc2317e5de2cc04443e24afb7b91ebd3fc1bd402
|
2022-09-27 14:31:34 +00:00 |
|
stephan
|
5b9973d898
|
WASM API renaming. Reworked JS API bootstrap's async post-init into a generic mechanism, no longer OPFS-specific.
FossilOrigin-Name: c42a8cb090cad1108dfd6be574202d744c59e053b505bc4c17252dc6b65d26bf
|
2022-09-27 13:40:12 +00:00 |
|
stephan
|
3d64548491
|
wasm/js: rename /persistent to /opfs to account for potential future persistent storage options. Minor flag-handling cleanups in the speedtest1 pages. Minor API tweaks in oo1.
FossilOrigin-Name: 4dc972a3656b2a9ec915bfb3f653136560c753ce4024c3f0d0d0c28f66db7a0a
|
2022-09-27 09:17:37 +00:00 |
|
stephan
|
278d3faf1f
|
Fiddle: replace db export routine with a C-side one which works for both Emscripten FS-hosted and OPFS-hosted db files. Minor code-adjacent cleanups.
FossilOrigin-Name: 3579a8d6f1f6cd3cd8aad9949536870c5fe7bae8c1778f700dd85d763e266b94
|
2022-09-26 13:55:10 +00:00 |
|
stephan
|
1f095d482d
|
Get fiddle db export working for OPFS VFS. Add root dir handle to the main OPFS VFS worker to enable creation of certain utility functions without delegating to the async worker. Add sqlite3.capi.sqlite3_wasm_rc_str() to map integer result codes back to their SQLITE_xxx counterparts. Minor doc touchups.
FossilOrigin-Name: 9b2244e1c8a40efe6547094a1b57acc8f2173145a8731abb0e36268ce0a8ef41
|
2022-09-26 11:38:58 +00:00 |
|
stephan
|
56fae744d4
|
Reworked out the OPFS async proxy metrics are fetched so that they play more nicely with the tight event-polling loop.
FossilOrigin-Name: ef503ced5c2ca842be9aea9ef13719a378ed3020e884032db09afee1b8eba0a1
|
2022-09-24 10:12:19 +00:00 |
|
stephan
|
e8afca3f16
|
Correct mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups.
FossilOrigin-Name: 86e2b55ec9483fa5add51a479c6509d73461f1ac6fca5d49e057b1c66f4314d2
|
2022-09-21 14:02:47 +00:00 |
|
stephan
|
72ab400d4d
|
Doc cleanups and additions. Add a way for the OPFS async runner to propagate exception text to the calling thread.
FossilOrigin-Name: 5c5e80652825cf883e6c17809cb98f2bf17d5feac2d263f6f492479154730dab
|
2022-09-21 12:27:35 +00:00 |
|
stephan
|
4cffb644c7
|
Correct a too-strict is-opfs-available check.
FossilOrigin-Name: 1b5f1b4a6c1457f98c258459e23e321fc59793de298fecb84031b87f02156cd5
|
2022-09-20 16:20:35 +00:00 |
|
stephan
|
b8c8d4e4a0
|
Speed up de/serialization of func args and return values in the OPFS VFS proxy.
FossilOrigin-Name: 5bf235bbe035e4ace7a54851e190742528af6b4266328a1b8bbb9fb3dd7f2118
|
2022-09-20 13:25:39 +00:00 |
|
stephan
|
c2ccd67689
|
OPFS proxy: remove one sanity-checking-only level of proxy function to shave off a few microseconds.
FossilOrigin-Name: b534831f3efb8910a17e29956e3e87cc80055ea66e15dbef992b6a556ff042f8
|
2022-09-20 10:47:36 +00:00 |
|
stephan
|
c9e2602e8f
|
Remove extraneous result-code SAB slots. Doc additions.
FossilOrigin-Name: 88de20712bbe3e11ea74af95edc34e9ab9f346f9aa0a30931e5a9e1f96ce57b0
|
2022-09-20 10:11:52 +00:00 |
|
stephan
|
5e8bb0aa13
|
An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS.
FossilOrigin-Name: a83ee3082d89439ea3ad5737e63e25bebb0f91895aca006ce5fecf5b93a2651a
|
2022-09-20 08:27:57 +00:00 |
|
stephan
|
138647a552
|
OPFS VFS: further internal refactoring towards experimenting with a new comms model.
FossilOrigin-Name: 5ca412ced24b4e3af5f467e710a597ed440badf7b8335346aade11d3cad3d1a1
|
2022-09-20 03:31:02 +00:00 |
|
stephan
|
c4b87be3e8
|
OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model.
FossilOrigin-Name: d4d63e4580ad8d497310608175308c03c517e051d7865cb66aa0b10356612d7d
|
2022-09-20 01:28:47 +00:00 |
|
stephan
|
aec046a264
|
Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect.
FossilOrigin-Name: d1f1fe6f1c60640f7770dfb9245c459a09b8d24ec2ddf664dff77c810bd51f96
|
2022-09-19 18:22:29 +00:00 |
|
stephan
|
f815011a08
|
Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th.
FossilOrigin-Name: fb7f287310d74a3e236125ae9c49b859f9263c29ae85161c1bcf9dd0778d8a51
|
2022-09-19 17:09:09 +00:00 |
|
stephan
|
0e0687ccfc
|
Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS).
FossilOrigin-Name: 2ec7e09139a510b9fd29e4c97283b20740a00f369193c6fecbb734f187e81b48
|
2022-09-19 13:44:23 +00:00 |
|
stephan
|
509f40526e
|
Generic cleanups in the OPFS VFS proxies.
FossilOrigin-Name: 7d5f6adc3b964413fc96ad8d2735312c3e58348024cabdd2099682cbf696eaf7
|
2022-09-19 09:58:01 +00:00 |
|
stephan
|
862281fc47
|
Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ.
FossilOrigin-Name: 56668f9902c6e896b6c63621a444444c6f58ee20f88a5feae97f1699be35892d
|
2022-09-19 09:25:25 +00:00 |
|
stephan
|
8766fd20d1
|
Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed).
FossilOrigin-Name: b9773f164878b0a1b7c88cc7a6d1374ea95f64920065e8b2b178a1afffd82fe5
|
2022-09-19 05:19:04 +00:00 |
|
stephan
|
f386012069
|
Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities.
FossilOrigin-Name: 26e625d05d9820033b23536f18ad3ddc59ed712ad507d4b0c7fe88abd15d2be8
|
2022-09-18 17:32:35 +00:00 |
|
stephan
|
c5313afea7
|
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client.
FossilOrigin-Name: 1c660970d0f62bcfd6e698a72b050d99972a1e39f45a5ac24194a190f8f78ab3
|
2022-09-18 02:35:30 +00:00 |
|