sqlite/ext/wasm
stephan 4cffb644c7 Correct a too-strict is-opfs-available check.
FossilOrigin-Name: 1b5f1b4a6c1457f98c258459e23e321fc59793de298fecb84031b87f02156cd5
2022-09-20 16:20:35 +00:00
..
api Correct a too-strict is-opfs-available check. 2022-09-20 16:20:35 +00:00
common Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th. 2022-09-19 17:09:09 +00:00
fiddle Remove the .timer script entry from fiddle app because emscripten has removed getrusage(), which breaks the timer. 2022-08-30 09:51:24 +00:00
jaccwabyt Corrected TextDecoder.decode() usage to run when its input references a SharedArrayBuffer. 2022-08-13 13:42:07 +00:00
sql Minor cleanups in OO API #1. Add Sudoku SQL to batch-runner.js's list. 2022-09-05 13:24:08 +00:00
batch-runner.html Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. 2022-09-13 19:27:03 +00:00
batch-runner.js Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. 2022-09-16 02:30:49 +00:00
demo-123-worker.html Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
demo-123.html Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
demo-123.js Minor doc improvements in demo-123.js. 2022-09-19 10:13:01 +00:00
demo-kvvfs1.html Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl. 2022-09-19 13:16:35 +00:00
demo-kvvfs1.js Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl. 2022-09-19 13:16:35 +00:00
EXPORTED_FUNCTIONS.fiddle More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00
EXPORTED_RUNTIME_METHODS.fiddle
GNUmakefile Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks. 2022-09-20 16:10:39 +00:00
index.html Remove an obsolete file. 2022-09-20 14:21:58 +00:00
README.md Add notes to ext/wasm/README.md explaining how to run tests on a remote 2022-08-18 15:53:27 +00:00
scratchpad-wasmfs-main.html Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 2022-09-15 03:09:00 +00:00
scratchpad-wasmfs-main.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
speedtest1-wasmfs.html Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number. 2022-09-19 14:56:13 +00:00
speedtest1-worker.html An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS. 2022-09-20 08:27:57 +00:00
speedtest1-worker.js Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th. 2022-09-19 17:09:09 +00:00
speedtest1.html Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks. 2022-09-20 16:10:39 +00:00
split-speedtest1-script.sh batch-runner.js: move generated SQL files into ./sql and start adding hand-written ones (like the Mandelbrot) to that set. 2022-08-30 10:04:08 +00:00
sqlite3-opfs-async-proxy.js Speed up de/serialization of func args and return values in the OPFS VFS proxy. 2022-09-20 13:25:39 +00:00
sqlite3-worker1-promiser.js Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
sqlite3-worker1.js Significant restructuring of the Worker #1 request/response object structures to improve readability and clarity. 2022-08-24 00:10:45 +00:00
test-opfs-vfs.html Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
test-opfs-vfs.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
testing1.html Get testing1.js working with a kvvfs build. 2022-09-12 22:27:00 +00:00
testing1.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
testing2.html Minor cleanups, reorgs, and doc updates for the JS APIs. Renamed sqlite3(-api)-worker.js to sqlite3(-api)-worker1.js, for symmetry with sqlite3-api-oo1.js. 2022-08-17 16:44:05 +00:00
testing2.js Consolidate oo1.DB.exec() and oo1.DB.execMulti() into oo1.DB.exec(). This is a bit less efficient but certainly easier for a client to deal with and lightens the maintenance burden. 2022-08-25 13:27:52 +00:00
testing-worker1-promiser.html More work on how to configure the sqlite3 JS API bootstrapping process from higher-level code. Initial version of sqlite3-worker1-promiser, a Promise-based proxy for the Worker API #1. 2022-08-24 05:59:23 +00:00
testing-worker1-promiser.js Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 2022-09-15 03:09:00 +00:00
wasmfs.make Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number. 2022-09-19 14:56:13 +00:00

This directory houses the Web Assembly (WASM) parts of the sqlite3 build.

It requires emscripten and that the build environment be set up for emscripten. A mini-HOWTO for setting that up follows...

First, install the Emscripten SDK, as documented here and summarized below for Linux environments:

# Clone the emscripten repository:
$ sudo apt install git
$ git clone https://github.com/emscripten-core/emsdk.git
$ cd emsdk

# Download and install the latest SDK tools:
$ ./emsdk install latest

# Make the "latest" SDK "active" for the current user:
$ ./emsdk activate latest

Those parts only need to be run once, but the SDK can be updated using:

$ git pull
$ ./emsdk activate latest

The following needs to be run for each shell instance which needs the emcc compiler:

# Activate PATH and other environment variables in the current terminal:
$ source ./emsdk_env.sh

$ which emcc
/path/to/emsdk/upstream/emscripten/emcc

Optionally, add that to your login shell's resource file (~/.bashrc or equivalent).

That env script needs to be sourced for building this application from the top of the sqlite3 build tree:

$ make fiddle

Or:

$ cd ext/wasm
$ make

That will generate the fiddle application under ext/fiddle, as fiddle.html. That application cannot, due to XMLHttpRequest security limitations, run if the HTML file is opened directly in the browser (i.e. if it is opened using a file:// URL), so it needs to be served via an HTTP server. For example, using althttpd:

$ cd ext/wasm/fiddle
$ althttpd -page fiddle.html

That will open the system's browser and run the fiddle app's page.

Note that when serving this app via althttpd, it must be a version from 2022-05-17 or newer so that it recognizes the .wasm file extension and responds with the mimetype application/wasm, as the WASM loader is pedantic about that detail.

Testing on a remote machine that is accessed via SSH

NB: The following are developer notes, last validated on 2022-08-18

  • Remote: Install git, emsdk, and althttpd
    • Use a version of althttpd that adds HTTP reply header lines to enable SharedArrayBuffers. These header lines are required:
            Cross-Origin-Opener-Policy: same-origin
            Cross-Origin-Embedder-Policy: require-corp
  • Remote: Install the SQLite source tree. CD to ext/wasm
  • Remote: "make" to build WASM
  • Remote: althttpd --port 8080 --popup
  • Local: ssh -L 8180:localhost:8080 remote
  • Local: Point your web-browser at http://localhost:8180/testing1.html

In order to enable SharedArrayBuffers, the web-browser requires that the two extra Cross-Origin lines be present in HTTP reply headers and that the request must come from "localhost". Since the web-server is on a different machine from the web-broser, the localhost requirement means that the connection must be tunneled using SSH.

Known Quirks and Limitations

Some "impedence mismatch" between C and WASM/JavaScript is to be expected.

No I/O

sqlite3 shell commands which require file I/O or pipes are disabled in the WASM build.

exit() Triggered from C

When C code calls exit(), as happens (for example) when running an "unsafe" command when safe mode is active, WASM's connection to the sqlite3 shell environment has no sensible choice but to shut down because exit() leaves it in a state we can no longer recover from. The JavaScript-side application attempts to recognize this and warn the user that restarting the application is necessary. Currently the only way to restart it is to reload the page. Restructuring the shell code such that it could be "rebooted" without restarting the JS app would require some invasive changes which are not currently on any TODO list but have not been entirely ruled out long-term.