| # sqlite3-api.js And Friends | |
| This is the README for the files `sqlite3-*.js` and | |
| `sqlite3-wasm.c`. This collection of files is used to build a | |
| single-file distribution of the sqlite3 JS/WASM API. It is broken into | |
| multiple JS files because: | |
| 1. To facilitate including or excluding certain components for | |
| specific use cases. e.g. by removing `sqlite3-api-oo1.js` if the | |
| OO#1 API is not needed. | |
| 2. To facilitate modularizing the pieces for use in different WASM | |
| build environments. e.g. the files `post-js-*.js` are for use with | |
| Emscripten's `--post-js` feature, and nowhere else. | |
| As-yet-hypothetical comparable toolchains would necessarily have | |
| similar facilities. | |
| 3. Certain components must be in their own standalone files in order | |
| to be loaded as JS Workers. | |
| The structure described here is the current state of things, as of | |
| this writing, but is not set in stone forever and may change at any | |
| time. This doc targets maintainers of this code and those wanting to | |
| dive in to the details, not end user. | |
| First off, a [pikchr][] of the proverbial onion: | |
| ```pikchr toggle center | |
| scale = 0.85 | |
| D0: dot invis | |
| define matryoshka { | |
| $rad = $rad + $2 | |
| circle with se at D0.c radius $rad | |
| text at last.nw $1 below ljust | |
| # $anchor = last circle.e | |
| } | |
| $rad = 5mm | |
| C0: circle with se at D0.c "sqlite3-api.js" fit | |
| $rad = C0.width / 2 | |
| matryoshka("--post-js",5mm) | |
| matryoshka("Emscripten",12mm) | |
| circle with nw at 7mm s of 3mm e of last circle.nw "--pre-js" fit | |
| matryoshka("--extern-pre/post-js",9mm) | |
| matryoshka("sqlite3.js",6mm) | |
| #CX: last circle | |
| right | |
| CW: file with w at 15mm e of 3rd circle.ne "sqlite3.wasm" fit | |
| arrow from CW.w to 3rd circle.ne | |
| CC: circle with e at 1cm w of last circle.w "End User" fit radius 1cm | |
| arrow <- from CC.e to 6th circle.w | |
| ``` | |
| Actually, `sqlite3.js` and `--extern-js` are the same. The former is | |
| what the client sees and the latter is how it looks from a code | |
| maintenance point of view. | |
| At the center of the onion is `sqlite3-api.js`, which gets generated | |
| by concatenating the following files together in their listed order: | |
| - **`sqlite3-api-prologue.js`** | |
| Contains the initial bootstrap setup of the sqlite3 API | |
| objects. This is exposed as a bootstrapping function so that | |
| the next step can pass in a config object which abstracts away parts | |
| of the WASM environment, to facilitate plugging it in to arbitrary | |
| WASM toolchains. The bootstrapping function gets removed from the | |
| global scope in a later stage of the bootstrapping process. | |
| - **`../common/whwasmutil.js`** | |
| A semi-third-party collection of JS/WASM utility code intended to | |
| replace much of the Emscripten glue. The sqlite3 APIs internally use | |
| these APIs instead of their Emscripten counterparts, in order to be | |
| more portable to arbitrary WASM toolchains and help reduce | |
| regressions caused by toolchain-level behavioral changes. This API | |
| is configurable, in principle, for use with arbitrary WASM | |
| toolchains. It is "semi-third-party" in that it was created in order | |
| to support this tree but is standalone and maintained together | |
| with... | |
| - **`../jaccwabyt/jaccwabyt.js`** | |
| Another semi-third-party API which creates bindings between JS | |
| and C structs, such that changes to the struct state from either JS | |
| or C are visible to the other end of the connection. This is also an | |
| independent spinoff project, conceived for the sqlite3 project but | |
| maintained separately. | |
| - **`sqlite3-api-glue.js`** | |
| Invokes functionality exposed by the previous two files to flesh out | |
| low-level parts of `sqlite3-api-prologue.js`. Most of these pieces | |
| involve populating the `sqlite3.capi.wasm` object and creating | |
| `sqlite3.capi.sqlite3_...()` bindings. This file also deletes most | |
| global-scope symbols the above files create, effectively moving them | |
| into the scope being used for initializing the API. | |
| - **`<build>/sqlite3-api-build-version.js`** | |
| Gets created by the build process and populates the | |
| `sqlite3.version` object. This part is not critical, but records the | |
| version of the library against which this module was built. | |
| - **`sqlite3-api-oo1.js`** | |
| Provides a high-level object-oriented wrapper to the lower-level C | |
| API, colloquially known as OO API #1. Its API is similar to other | |
| high-level sqlite3 JS wrappers and should feel relatively familiar | |
| to anyone familiar with such APIs. It is not a "required component" | |
| and can be elided from builds which do not want it. | |
| - **`sqlite3-api-worker1.js`** | |
| A Worker-thread-based API which uses OO API #1 to provide an | |
| interface to a database which can be driven from the main Window | |
| thread via the Worker message-passing interface. Like OO API #1, | |
| this is an optional component, offering one of any number of | |
| potential implementations for such an API. | |
| - **`sqlite3-worker1.js`** | |
| Is not part of the amalgamated sources and is intended to be | |
| loaded by a client Worker thread. It loads the sqlite3 module | |
| and runs the Worker #1 API which is implemented in | |
| `sqlite3-api-worker1.js`. | |
| - **`sqlite3-worker1-promiser.js`** | |
| Is likewise not part of the amalgamated sources and provides | |
| a Promise-based interface into the Worker #1 API. This is | |
| a far user-friendlier way to interface with databases running | |
| in a Worker thread. | |
| - **`sqlite3-vfs-helper.c-pp.js`** | |
| Installs the `sqlite3.vfs` namespace, which contain helpers for use | |
| by downstream code which creates `sqlite3_vfs` implementations. | |
| - **`sqlite3-vtab-helper.c-pp.js`** | |
| Installs the `sqlite3.vtab` namespace, which contain helpers for use | |
| by downstream code which creates `sqlite3_module` implementations. | |
| - **`sqlite3-vfs-opfs.c-pp.js`** | |
| is an sqlite3 VFS implementation which supports the [Origin-Private | |
| FileSystem (OPFS)][OPFS] as a storage layer to provide persistent | |
| storage for database files in a browser. It requires... | |
| - **`sqlite3-opfs-async-proxy.js`** | |
| is the asynchronous backend part of the [OPFS][] proxy. It | |
| speaks directly to the (async) OPFS API and channels those | |
| results back to its synchronous counterpart. This file, because | |
| it must be started in its own Worker, is not part of the | |
| amalgamation. | |
| - **`sqlite3-vfs-opfs-sahpool.c-pp.js`** | |
| is another sqlite3 VFS supporting the [OPFS][], but uses a | |
| completely different approach than the above-listed one. | |
| The previous files do not immediately extend the library. Instead they | |
| install a global function `sqlite3ApiBootstrap()`, which downstream | |
| code must call to configure the library for the current JS/WASM | |
| environment. Each file listed above pushes a callback into the | |
| bootstrapping queue, to be called as part of `sqlite3ApiBootstrap()`. | |
| Some files also temporarily create global objects in order to | |
| communicate their state to the files which follow them. Those | |
| get cleaned up vi `post-js-footer.js`, described below. | |
| Adapting the build for non-Emscripten toolchains essentially requires packaging | |
| the above files, concatated together, into that toolchain's "JS glue" | |
| and, in the final stage of that glue, call `sqlite3ApiBootstrap()` and | |
| return its result to the end user. | |
| **Files with the extension `.c-pp.js`** are intended [to be processed | |
| with `c-pp`](#c-pp), noting that such preprocessing may be applied | |
| after all of the relevant files are concatenated. The `.c-pp.js` | |
| extension is used primarily to keep the code maintainers cognisant of | |
| the fact that those files contain constructs which may not run as-is | |
| in any given JavaScript environment. | |
| The build process glues those files together, resulting in | |
| `sqlite3-api.js`, which is everything except for the Emscripten-specific | |
| files detailed below (into which `sqlite3-api.js` gets injected). | |
| The non-JS outlier file is `sqlite3-wasm.c`: it is a proxy for | |
| `sqlite3.c` which `#include`'s that file and adds a handful of | |
| WASM-specific helper functions, at least two of which requires access | |
| to private/static `sqlite3.c` internals. `sqlite3.wasm` is compiled | |
| from this file rather than `sqlite3.c`. | |
| The following Emscripten-specific files are injected into the | |
| build-generated `sqlite3.js` along with `sqlite3-api.js`. | |
| - **`extern-pre-js.js`** | |
| Emscripten-specific header for Emscripten's `--extern-pre-js` | |
| flag. As of this writing, that file is only used for experimentation | |
| purposes and holds no code relevant to the production deliverables. | |
| - **`pre-js.c-pp.js`** | |
| Emscripten-specific header for Emscripten's `--pre-js` flag. This | |
| file overrides certain Emscripten behavior before Emscripten does | |
| most of its work. | |
| - **`post-js-header.js`** | |
| Emscripten-specific header for the `--post-js` input. It opens up, | |
| but does not close, a function used for initializing the library. | |
| - **`sqlite3-api.js`** gets sandwiched between these ↑ two | |
| ↓ files. | |
| - **`post-js-footer.js`** | |
| Emscripten-specific footer for the `--post-js` input. This closes | |
| off the function opened by `post-js-header.js`. This file cleans up | |
| any dangling globals and runs `sqlite3ApiBootstrap()`. As of this | |
| writing, this code ensures that the previous files leave no more | |
| than a single global symbol installed - `sqlite3InitModule()`. | |
| - **`extern-post-js.c-pp.js`** | |
| Emscripten-specific header for Emscripten's `--extern-post-js` | |
| flag. This file is run in the global scope. It overwrites the | |
| Emscripten-installed `sqlite3InitModule()` function with one which | |
| first runs the original implementation, then runs the function | |
| installed by `post-js-header/footer.js` to initialize the library, | |
| then initializes the asynchronous parts of the sqlite3 module (for | |
| example, the [OPFS][] VFS support). Its final step is to return a | |
| Promise of the sqlite3 namespace object to the caller of | |
| `sqlite3InitModule()` (the library user). | |
| <a id='c-pp'></a> | |
| Preprocessing of Source Files | |
| ------------------------------------------------------------------------ | |
| Certain files in the build require preprocessing to filter in/out | |
| parts which differ between vanilla JS, ES6 Modules, and node.js | |
| builds. The preprocessor application itself is in | |
| [`c-pp-lite.c`](/file/ext/wasm/c-pp-lite.c) and the complete technical | |
| details of such preprocessing are maintained in | |
| [`GNUMakefile`](/file/ext/wasm/GNUmakefile). | |
| [OPFS]: https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system | |
| [pikchr]: https://pikchr.org | |