sqlite/ext/wasm
stephan ac51eb7754 Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes.
FossilOrigin-Name: 2e4a005bd35424caeaa99ace23162cf79e2ebdb159475ffad92b85dc864ad764
2022-09-19 03:57:31 +00:00
..
api Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
common speedtest1 wasm: add a link in the worker variant which launches the main-thread variant with the selected CLI flags. Append collected stdout/stderr to the main-thread page after execution is finished to avoid having to open the dev tools (which inexplicably slows down wasm execution). 2022-09-09 04:50:18 +00:00
fiddle Remove the .timer script entry from fiddle app because emscripten has removed getrusage(), which breaks the timer. 2022-08-30 09:51:24 +00:00
jaccwabyt Corrected TextDecoder.decode() usage to run when its input references a SharedArrayBuffer. 2022-08-13 13:42:07 +00:00
sql Minor cleanups in OO API #1. Add Sudoku SQL to batch-runner.js's list. 2022-09-05 13:24:08 +00:00
batch-runner-kvvfs.html Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. 2022-09-16 02:30:49 +00:00
batch-runner.html Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. 2022-09-13 19:27:03 +00:00
batch-runner.js Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. 2022-09-16 02:30:49 +00:00
demo-123-worker.html Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
demo-123.html Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
demo-123.js Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
EXPORTED_FUNCTIONS.fiddle More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00
EXPORTED_RUNTIME_METHODS.fiddle wasm/fiddle refactoring part 1 of N: move fiddle app from ext/fiddle to ext/wasm/fiddle, which only contains files intended to be pushed to the live site. Disabled build of the non-fiddle wasm parts, pending a later step of the refactoring. 2022-08-10 09:36:10 +00:00
GNUmakefile Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there. 2022-09-19 00:40:53 +00:00
index.html Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
kvvfs1.html Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. 2022-09-13 19:27:03 +00:00
kvvfs1.js Clean up some JS documentation. 2022-09-15 02:27:48 +00:00
kvvfs.make Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7. 2022-09-16 01:05:19 +00:00
README.md Add notes to ext/wasm/README.md explaining how to run tests on a remote 2022-08-18 15:53:27 +00:00
scratchpad-wasmfs-main.html Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 2022-09-15 03:09:00 +00:00
scratchpad-wasmfs-main.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
speedtest1-kvvfs.html Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit. 2022-09-16 20:16:50 +00:00
speedtest1-wasmfs.html More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00
speedtest1-worker.html Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7. 2022-09-16 01:05:19 +00:00
speedtest1-worker.js More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00
speedtest1.html More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00
split-speedtest1-script.sh batch-runner.js: move generated SQL files into ./sql and start adding hand-written ones (like the Mandelbrot) to that set. 2022-08-30 10:04:08 +00:00
sqlite3-opfs-async-proxy.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
sqlite3-worker1-promiser.js Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. 2022-09-19 03:57:31 +00:00
sqlite3-worker1.js Significant restructuring of the Worker #1 request/response object structures to improve readability and clarity. 2022-08-24 00:10:45 +00:00
test-opfs-vfs.html Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
test-opfs-vfs.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
testing1.html Get testing1.js working with a kvvfs build. 2022-09-12 22:27:00 +00:00
testing1.js Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. 2022-09-18 17:32:35 +00:00
testing2.html Minor cleanups, reorgs, and doc updates for the JS APIs. Renamed sqlite3(-api)-worker.js to sqlite3(-api)-worker1.js, for symmetry with sqlite3-api-oo1.js. 2022-08-17 16:44:05 +00:00
testing2.js Consolidate oo1.DB.exec() and oo1.DB.execMulti() into oo1.DB.exec(). This is a bit less efficient but certainly easier for a client to deal with and lightens the maintenance burden. 2022-08-25 13:27:52 +00:00
testing-worker1-promiser.html More work on how to configure the sqlite3 JS API bootstrapping process from higher-level code. Initial version of sqlite3-worker1-promiser, a Promise-based proxy for the Worker API #1. 2022-08-24 05:59:23 +00:00
testing-worker1-promiser.js Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. 2022-09-15 03:09:00 +00:00
wasmfs.make More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. 2022-09-15 06:42:41 +00:00

This directory houses the Web Assembly (WASM) parts of the sqlite3 build.

It requires emscripten and that the build environment be set up for emscripten. A mini-HOWTO for setting that up follows...

First, install the Emscripten SDK, as documented here and summarized below for Linux environments:

# Clone the emscripten repository:
$ sudo apt install git
$ git clone https://github.com/emscripten-core/emsdk.git
$ cd emsdk

# Download and install the latest SDK tools:
$ ./emsdk install latest

# Make the "latest" SDK "active" for the current user:
$ ./emsdk activate latest

Those parts only need to be run once, but the SDK can be updated using:

$ git pull
$ ./emsdk activate latest

The following needs to be run for each shell instance which needs the emcc compiler:

# Activate PATH and other environment variables in the current terminal:
$ source ./emsdk_env.sh

$ which emcc
/path/to/emsdk/upstream/emscripten/emcc

Optionally, add that to your login shell's resource file (~/.bashrc or equivalent).

That env script needs to be sourced for building this application from the top of the sqlite3 build tree:

$ make fiddle

Or:

$ cd ext/wasm
$ make

That will generate the fiddle application under ext/fiddle, as fiddle.html. That application cannot, due to XMLHttpRequest security limitations, run if the HTML file is opened directly in the browser (i.e. if it is opened using a file:// URL), so it needs to be served via an HTTP server. For example, using althttpd:

$ cd ext/wasm/fiddle
$ althttpd -page fiddle.html

That will open the system's browser and run the fiddle app's page.

Note that when serving this app via althttpd, it must be a version from 2022-05-17 or newer so that it recognizes the .wasm file extension and responds with the mimetype application/wasm, as the WASM loader is pedantic about that detail.

Testing on a remote machine that is accessed via SSH

NB: The following are developer notes, last validated on 2022-08-18

  • Remote: Install git, emsdk, and althttpd
    • Use a version of althttpd that adds HTTP reply header lines to enable SharedArrayBuffers. These header lines are required:
            Cross-Origin-Opener-Policy: same-origin
            Cross-Origin-Embedder-Policy: require-corp
  • Remote: Install the SQLite source tree. CD to ext/wasm
  • Remote: "make" to build WASM
  • Remote: althttpd --port 8080 --popup
  • Local: ssh -L 8180:localhost:8080 remote
  • Local: Point your web-browser at http://localhost:8180/testing1.html

In order to enable SharedArrayBuffers, the web-browser requires that the two extra Cross-Origin lines be present in HTTP reply headers and that the request must come from "localhost". Since the web-server is on a different machine from the web-broser, the localhost requirement means that the connection must be tunneled using SSH.

Known Quirks and Limitations

Some "impedence mismatch" between C and WASM/JavaScript is to be expected.

No I/O

sqlite3 shell commands which require file I/O or pipes are disabled in the WASM build.

exit() Triggered from C

When C code calls exit(), as happens (for example) when running an "unsafe" command when safe mode is active, WASM's connection to the sqlite3 shell environment has no sensible choice but to shut down because exit() leaves it in a state we can no longer recover from. The JavaScript-side application attempts to recognize this and warn the user that restarting the application is necessary. Currently the only way to restart it is to reload the page. Restructuring the shell code such that it could be "rebooted" without restarting the JS app would require some invasive changes which are not currently on any TODO list but have not been entirely ruled out long-term.