sqlite/ext/wasm
stephan dd628ed58b Make use of the --big-transactions flag in the speedtest1 JS apps.
FossilOrigin-Name: f2846dcbcaac7880394fb14597c3a60ed310419128c4c5b863cd771a7e5cdeb5
2022-09-08 21:33:50 +00:00
..
api Get speedtest1.js working with WASMFS/OPFS. 2022-09-06 20:17:15 +00:00
common Add speedtest1-worker.html, an interactive Worker-thread variant of speedtest1.html. Add ext/wasm/index.html to act as a gateway to the various test pages. 2022-09-08 15:30:59 +00:00
fiddle Remove the .timer script entry from fiddle app because emscripten has removed getrusage(), which breaks the timer. 2022-08-30 09:51:24 +00:00
jaccwabyt Corrected TextDecoder.decode() usage to run when its input references a SharedArrayBuffer. 2022-08-13 13:42:07 +00:00
sql Minor cleanups in OO API #1. Add Sudoku SQL to batch-runner.js's list. 2022-09-05 13:24:08 +00:00
batch-runner.html Add speedtest1-worker.html, an interactive Worker-thread variant of speedtest1.html. Add ext/wasm/index.html to act as a gateway to the various test pages. 2022-09-08 15:30:59 +00:00
batch-runner.js wasm: minor text and build cleanups. 2022-09-03 11:41:44 +00:00
demo-oo1.html wasm: add a small demo/presentation app for JS OO API #1 and make a few minor additions to that API. 2022-08-16 16:36:19 +00:00
demo-oo1.js Consolidate oo1.DB.exec() and oo1.DB.execMulti() into oo1.DB.exec(). This is a bit less efficient but certainly easier for a client to deal with and lightens the maintenance burden. 2022-08-25 13:27:52 +00:00
EXPORTED_FUNCTIONS.fiddle Build fiddle with WASMFS OPFS support and attempt to use it if available. It does not work because of an inexplicable exception in Emscripten-generated code and perpetually-locked db, but it's not yet clear why. 2022-08-12 17:57:09 +00:00
EXPORTED_RUNTIME_METHODS.fiddle wasm/fiddle refactoring part 1 of N: move fiddle app from ext/fiddle to ext/wasm/fiddle, which only contains files intended to be pushed to the live site. Disabled build of the non-fiddle wasm parts, pending a later step of the refactoring. 2022-08-10 09:36:10 +00:00
GNUmakefile Minor build cleanups and tweaks in the speedtest1 wasm apps. 2022-09-08 20:04:52 +00:00
index.html Add speedtest1-worker.html, an interactive Worker-thread variant of speedtest1.html. Add ext/wasm/index.html to act as a gateway to the various test pages. 2022-09-08 15:30:59 +00:00
README.md Add notes to ext/wasm/README.md explaining how to run tests on a remote 2022-08-18 15:53:27 +00:00
scratchpad-opfs-main.html wasm: accommodate moving generated sqlite3.js in the previous checkin. 2022-08-16 16:11:20 +00:00
scratchpad-opfs-main.js wasm: accommodated a JS API rename. 2022-08-22 08:55:10 +00:00
scratchpad-opfs-worker2.js wasm: accommodated a JS API rename. 2022-08-22 08:55:10 +00:00
scratchpad-opfs-worker.html Add worker-style variant of the tests added in [ae24ac0f7dd9], but building this with wasmfs causes them to throw inexplicable exceptions from the Emscripten glue (without wasmfs it builds and runs fine, but storage is not persistent). 2022-08-13 17:13:16 +00:00
scratchpad-opfs-worker.js wasm: accommodate moving generated sqlite3.js in the previous checkin. 2022-08-16 16:11:20 +00:00
speedtest1-worker.html Make use of the --big-transactions flag in the speedtest1 JS apps. 2022-09-08 21:33:50 +00:00
speedtest1-worker.js Add speedtest1-worker.html, an interactive Worker-thread variant of speedtest1.html. Add ext/wasm/index.html to act as a gateway to the various test pages. 2022-09-08 15:30:59 +00:00
speedtest1.html Make use of the --big-transactions flag in the speedtest1 JS apps. 2022-09-08 21:33:50 +00:00
split-speedtest1-script.sh batch-runner.js: move generated SQL files into ./sql and start adding hand-written ones (like the Mandelbrot) to that set. 2022-08-30 10:04:08 +00:00
sqlite3-worker1-promiser.js Expand the worker1 'exec' op handling for per-row callbacks for API-level consistency and smooth some edges between worker1 core and worker1-promiser. Add worker1 'config-get' message to fetch the serializable parts of the sqlite3.config state. Improve the 'open' op's handling of the 'persistent' option (noting that we cannot yet test that case from a worker). 2022-08-24 18:39:46 +00:00
sqlite3-worker1.js Significant restructuring of the Worker #1 request/response object structures to improve readability and clarity. 2022-08-24 00:10:45 +00:00
testing1.html wasm: add a small demo/presentation app for JS OO API #1 and make a few minor additions to that API. 2022-08-16 16:36:19 +00:00
testing1.js More experimentation with how client-side config state can be passed on to initialize the JS-side sqlite3 API. 2022-08-22 21:37:17 +00:00
testing2.html Minor cleanups, reorgs, and doc updates for the JS APIs. Renamed sqlite3(-api)-worker.js to sqlite3(-api)-worker1.js, for symmetry with sqlite3-api-oo1.js. 2022-08-17 16:44:05 +00:00
testing2.js Consolidate oo1.DB.exec() and oo1.DB.execMulti() into oo1.DB.exec(). This is a bit less efficient but certainly easier for a client to deal with and lightens the maintenance burden. 2022-08-25 13:27:52 +00:00
testing-worker1-promiser.html More work on how to configure the sqlite3 JS API bootstrapping process from higher-level code. Initial version of sqlite3-worker1-promiser, a Promise-based proxy for the Worker API #1. 2022-08-24 05:59:23 +00:00
testing-worker1-promiser.js Refactor and expand the worker1 docs, consolidating them into the top of their file instead of scattered around the internals. Accommodate an API change from yesterday in demo-oo1.js. 2022-08-25 11:39:12 +00:00

This directory houses the Web Assembly (WASM) parts of the sqlite3 build.

It requires emscripten and that the build environment be set up for emscripten. A mini-HOWTO for setting that up follows...

First, install the Emscripten SDK, as documented here and summarized below for Linux environments:

# Clone the emscripten repository:
$ sudo apt install git
$ git clone https://github.com/emscripten-core/emsdk.git
$ cd emsdk

# Download and install the latest SDK tools:
$ ./emsdk install latest

# Make the "latest" SDK "active" for the current user:
$ ./emsdk activate latest

Those parts only need to be run once, but the SDK can be updated using:

$ git pull
$ ./emsdk activate latest

The following needs to be run for each shell instance which needs the emcc compiler:

# Activate PATH and other environment variables in the current terminal:
$ source ./emsdk_env.sh

$ which emcc
/path/to/emsdk/upstream/emscripten/emcc

Optionally, add that to your login shell's resource file (~/.bashrc or equivalent).

That env script needs to be sourced for building this application from the top of the sqlite3 build tree:

$ make fiddle

Or:

$ cd ext/wasm
$ make

That will generate the fiddle application under ext/fiddle, as fiddle.html. That application cannot, due to XMLHttpRequest security limitations, run if the HTML file is opened directly in the browser (i.e. if it is opened using a file:// URL), so it needs to be served via an HTTP server. For example, using althttpd:

$ cd ext/wasm/fiddle
$ althttpd -page fiddle.html

That will open the system's browser and run the fiddle app's page.

Note that when serving this app via althttpd, it must be a version from 2022-05-17 or newer so that it recognizes the .wasm file extension and responds with the mimetype application/wasm, as the WASM loader is pedantic about that detail.

Testing on a remote machine that is accessed via SSH

NB: The following are developer notes, last validated on 2022-08-18

  • Remote: Install git, emsdk, and althttpd
    • Use a version of althttpd that adds HTTP reply header lines to enable SharedArrayBuffers. These header lines are required:
            Cross-Origin-Opener-Policy: same-origin
            Cross-Origin-Embedder-Policy: require-corp
  • Remote: Install the SQLite source tree. CD to ext/wasm
  • Remote: "make" to build WASM
  • Remote: althttpd --port 8080 --popup
  • Local: ssh -L 8180:localhost:8080 remote
  • Local: Point your web-browser at http://localhost:8180/testing1.html

In order to enable SharedArrayBuffers, the web-browser requires that the two extra Cross-Origin lines be present in HTTP reply headers and that the request must come from "localhost". Since the web-server is on a different machine from the web-broser, the localhost requirement means that the connection must be tunneled using SSH.

Known Quirks and Limitations

Some "impedence mismatch" between C and WASM/JavaScript is to be expected.

No I/O

sqlite3 shell commands which require file I/O or pipes are disabled in the WASM build.

exit() Triggered from C

When C code calls exit(), as happens (for example) when running an "unsafe" command when safe mode is active, WASM's connection to the sqlite3 shell environment has no sensible choice but to shut down because exit() leaves it in a state we can no longer recover from. The JavaScript-side application attempts to recognize this and warn the user that restarting the application is necessary. Currently the only way to restart it is to reload the page. Restructuring the shell code such that it could be "rebooted" without restarting the JS app would require some invasive changes which are not currently on any TODO list but have not been entirely ruled out long-term.