mirror of
https://github.com/mountain-loop/yaak.git
synced 2026-08-22 11:23:59 +02:00
Run plugins in a QuickJS sandbox in the browser
Adds packages/plugin-sandbox: QuickJS-ng compiled to wasm, running in a dedicated worker, with a runtime shell inside it that loads a plugin bundle and answers the same InternalEventPayload events the Node runtime answers. Plugins are unmodified. Wires the browser host's template function, authentication, cURL import and template render commands to it, and relaxes TemplateCallback's Send bound on wasm32 so the engine's renderer can call back out to a plugin.
This commit is contained in:
Gregory Schier
@@ -0,0 +1,275 @@
|
||||
# The Yaak plugin sandbox
|
||||
|
||||
A QuickJS interpreter, a small set of globals, and one function that calls the
|
||||
host. That is the whole runtime. Everything else a plugin does — read a request,
|
||||
send one, store a token, ask the user something — is a message the host chose to
|
||||
answer.
|
||||
|
||||
This document is the contract. It is written to be implementable twice: once
|
||||
here, in wasm, for the browser, and once in Rust with `rquickjs`, for the desktop
|
||||
and the CLI. **If the two hosts disagree about anything below, that is a bug in
|
||||
whichever one drifted, not a platform difference to work around.** The promise
|
||||
to plugin authors is that there is one sandbox and it behaves the same
|
||||
everywhere; a promise like that is only worth making if it is enforceable, which
|
||||
is why the browser runs QuickJS rather than the Worker's own JavaScript engine.
|
||||
|
||||
## The engine
|
||||
|
||||
**quickjs-ng**, and only quickjs-ng.
|
||||
|
||||
There is no real choice: `rquickjs` — the Rust binding the desktop host will use
|
||||
— vendors quickjs-ng as a git submodule and offers no alternative. Picking
|
||||
Bellard's upstream for the browser would mean the two hosts run different
|
||||
engines, which is exactly the thing this design exists to prevent.
|
||||
|
||||
| | Version | Notes |
|
||||
|---|---|---|
|
||||
| Browser (this package) | quickjs-ng **0.12.1** | via `@jitl/quickjs-ng-wasmfile-release-sync` 0.32.0 |
|
||||
| Desktop (planned) | quickjs-ng **0.15.1** | via `rquickjs` 0.12.2 |
|
||||
|
||||
**The version skew is a known gap, and closing it is slice-2 work.** Three minor
|
||||
versions is small — the differences are bug fixes and `Temporal` progress, not
|
||||
semantics anything here depends on — but "identical everywhere" is not a claim
|
||||
that survives being approximate indefinitely. Whoever builds the Rust host
|
||||
should pin both sides to the same tag and add a test that asserts the version
|
||||
string matches.
|
||||
|
||||
### Why the sync build, not ASYNCIFY
|
||||
|
||||
`quickjs-emscripten` ships an ASYNCIFY variant that lets guest code call an async
|
||||
host function *synchronously*. We use the plain sync build instead:
|
||||
|
||||
- ASYNCIFY is about twice the wasm size (1.08 MB vs 529 KB) and, measured,
|
||||
**2.2x slower**.
|
||||
- It can only suspend for one host call at a time. A runtime that runs several
|
||||
plugins would have to hold one wasm instance per in-flight call.
|
||||
- We do not need it. The guest gets real `await` anyway: a host function returns
|
||||
a QuickJS deferred promise, the host resolves it, and the host drains the job
|
||||
queue. `ctx.store.get(...)` is an ordinary `await` inside a plugin.
|
||||
|
||||
The only thing lost is a host call that *looks* synchronous to the guest, and no
|
||||
Yaak plugin wants one — the whole `ctx` API has been async since it existed.
|
||||
|
||||
## What exists inside the sandbox
|
||||
|
||||
QuickJS gives you the language and nothing else. Everything below is either
|
||||
installed by `src/guest/globals.ts` or absent. **Both hosts must install exactly
|
||||
this list.**
|
||||
|
||||
### From the engine
|
||||
|
||||
`Object`, `Array`, `Function`, `String`, `Number`, `Boolean`, `Symbol`, `Math`,
|
||||
`JSON`, `Date`, `RegExp`, `Error` and subclasses, `Map`, `Set`, `WeakMap`,
|
||||
`WeakSet`, `WeakRef`, `Promise`, `Proxy`, `Reflect`, `BigInt`, `ArrayBuffer`,
|
||||
`SharedArrayBuffer`, `DataView`, all `TypedArray`s, `globalThis`,
|
||||
`queueMicrotask`, `performance`.
|
||||
|
||||
Language level is ES2023 plus most of ES2024 — `Object.groupBy`,
|
||||
`Array.prototype.at`, `String.prototype.replaceAll`, async generators, private
|
||||
fields, `??=` all work.
|
||||
|
||||
### Installed by the runtime
|
||||
|
||||
| Global | Notes |
|
||||
|---|---|
|
||||
| `console` | `.log/.info/.warn/.error/.debug/.trace`. Arguments are formatted to a string **inside** the sandbox, so only strings cross out — a cycle or an exotic prototype is the guest's problem, not the host's. |
|
||||
| `setTimeout` / `clearTimeout` | The host holds the real timer; QuickJS has no clock to wake on. A sandbox torn down mid-wait takes its pending timers with it. |
|
||||
| `TextEncoder` / `TextDecoder` | UTF-8 only. Pure JavaScript, in-sandbox — a bridge would cost a copy each way. Lone surrogates encode to U+FFFD, matching the standard. |
|
||||
| `btoa` / `atob` | Latin-1, same narrow contract as the browser's. |
|
||||
|
||||
### Deliberately absent
|
||||
|
||||
`fetch`, `XMLHttpRequest`, `WebSocket`, `crypto`, `structuredClone`, `URL`,
|
||||
`URLSearchParams`, `setInterval`, `require`, `module`, `process`, `Buffer`,
|
||||
`std`, `os`, and every Node built-in.
|
||||
|
||||
- **Network and storage are absent because they are `ctx`'s job.** A plugin that
|
||||
could open its own socket would defeat the point of the sandbox and would not
|
||||
work in a browser anyway.
|
||||
- **`setInterval` is absent** because an interval is a timer that rearms and
|
||||
nothing in a plugin should be polling. Build one from `setTimeout`, visibly.
|
||||
- **`crypto` is absent, and this is the one real gap.** The decided direction is
|
||||
pure-JavaScript `@noble/*` inside the sandbox: audited, dependency-free,
|
||||
identical on both hosts, no host API to keep in sync. A `yaak.crypto` builtin
|
||||
is the escape hatch **if** a hot path is measured, not before. Concretely,
|
||||
`template-function-uuid` does not run in the sandbox today because its `uuid`
|
||||
dependency reaches for `node:crypto`; that is a slice-2 conversion, not a
|
||||
missing capability.
|
||||
- **`URL` is absent** only because nothing has needed it yet. It is a reasonable
|
||||
future addition; it must be added to both hosts together.
|
||||
|
||||
## The module contract
|
||||
|
||||
A module arrives as **source text**, not a file — there is no filesystem, and in
|
||||
a browser there could not be one.
|
||||
|
||||
It is evaluated as CommonJS, via `new Function("module", "exports", "require", source)`,
|
||||
and must assign `module.exports.plugin` (or `module.exports.default`). `new
|
||||
Function` rather than an ES module is deliberate: the bundle's top-level names
|
||||
cannot collide with the shell's, and the source needs no loader hook.
|
||||
|
||||
`require` exists **only to throw**, naming the specifier. A bundle that still
|
||||
calls it was not bundled for this target, and saying which module is missing
|
||||
beats an `undefined` that surfaces ten frames later.
|
||||
|
||||
Bundling requirements: CommonJS, no external modules, no Node built-ins, ES2022.
|
||||
`scripts/bundle-sandbox-plugins.mjs` does this today; what a real
|
||||
`yaakcli build --target sandbox` needs is listed at the bottom of that file.
|
||||
|
||||
## The host interface
|
||||
|
||||
Four functions, installed on `globalThis` before any plugin code runs. A Rust
|
||||
host must expose the same four with the same names and shapes.
|
||||
|
||||
| Function | Direction | Shape |
|
||||
|---|---|---|
|
||||
| `__yaak_call(envelopeJson)` | guest → host | Returns a **promise** of the reply JSON. The one door out. |
|
||||
| `__yaak_log(level, message)` | guest → host | Both strings. Fire and forget. |
|
||||
| `__yaak_timer_start(id, ms)` | guest → host | Host calls `__yaak_guest.fireTimer(id)` when due. |
|
||||
| `__yaak_timer_cancel(id)` | guest → host | |
|
||||
|
||||
And the guest exposes `globalThis.__yaak_guest`:
|
||||
|
||||
| Method | Shape |
|
||||
|---|---|
|
||||
| `load(source, pluginRefId)` | Evaluate a module. Throws if it exports no `plugin`. |
|
||||
| `summary()` | What the module contributes, as plain data. |
|
||||
| `dispatch(envelopeJson)` | Returns a promise of the reply payload JSON. |
|
||||
| `fireTimer(id)` | |
|
||||
|
||||
### Envelopes
|
||||
|
||||
Both directions carry `InternalEventPayload` from
|
||||
`crates/yaak-plugins/src/events.rs`, **unchanged**. That is what makes a plugin
|
||||
unable to tell which runtime it is in.
|
||||
|
||||
```jsonc
|
||||
// dispatch, host → guest
|
||||
{ "context": { "id": "...", "label": null, "workspaceId": "..." },
|
||||
"payload": { "type": "call_template_function_request", "name": "...", "args": { ... } } }
|
||||
|
||||
// __yaak_call, guest → host
|
||||
{ "pluginRefId": "auth-bearer",
|
||||
"context": { ... },
|
||||
"payload": { "type": "get_key_value_request", "key": "token" } }
|
||||
```
|
||||
|
||||
`pluginRefId` rides on outgoing calls because one host handler serves every
|
||||
loaded module, and a plugin's stored state is namespaced by which plugin it is —
|
||||
the same namespacing `build_shared_reply` does in `crates/yaak/src/plugin_events.rs`.
|
||||
|
||||
A throw inside a plugin becomes `{"type":"error_response","error":"..."}`, never
|
||||
a crash and never silence: whatever asked gets a message.
|
||||
|
||||
## The `ctx` API
|
||||
|
||||
Built entirely out of `__yaak_call`. See `src/guest/context.ts` — it is the same
|
||||
surface the Node runtime's `PluginInstance` builds, so it is not repeated here.
|
||||
|
||||
What differs is which calls a **host** answers. The browser host answers a
|
||||
deliberately short list (`packages/platform/src/web/plugins.ts`) and refuses the
|
||||
rest by name. Refusing by name matters: a plugin that needs something it cannot
|
||||
have should fail with a sentence someone can act on.
|
||||
|
||||
Answered in the browser today: `get_key_value`, `set_key_value`,
|
||||
`delete_key_value`, `show_toast`. Everything else — sends, model reads and
|
||||
writes, prompts, response bodies, window info — refuses. Those are capability
|
||||
decisions, not oversights, and each should be added one at a time.
|
||||
|
||||
`ctx.window.openUrl` throws in *every* sandbox host: a plugin-opened window is a
|
||||
desktop affordance with no browser equivalent, and handing back a handle whose
|
||||
`close()` does nothing would be worse.
|
||||
|
||||
## Isolation and limits
|
||||
|
||||
One runtime per worker, **one context per module**. A context is the isolation
|
||||
boundary — its own globals, its own `Object`, its own prototypes — so two plugins
|
||||
cannot see or patch each other. Sharing the runtime is deliberate: the engine and
|
||||
its wasm instance are the expensive part; contexts are not.
|
||||
|
||||
| Limit | Value | Why |
|
||||
|---|---|---|
|
||||
| Memory | 256 MB per runtime | Sized for an importer holding a large document and the objects it parses into. |
|
||||
| Stack | 2 MB | Deep recursion becomes a guest stack overflow, not a worker crash. |
|
||||
| Synchronous execution | 60 s | A watchdog for `while (true)`, **not** a limit on real work. |
|
||||
|
||||
The watchdog bounds *synchronous* execution only. A plugin awaiting the host is
|
||||
not looping, so the clock stops for the duration of a host call and restarts
|
||||
when the guest resumes. It is generous because it costs nothing to be: plugins
|
||||
run in their own worker, so one stuck there blocks no database command and no
|
||||
frame. It is sized off the slowest real work measured — GitHub's 12.3 MB OpenAPI
|
||||
description takes about 2.5 s (`bench/import.mjs`) — with room for a document
|
||||
several times larger before a legitimate import looks like a hang.
|
||||
|
||||
## Where the sandbox runs, and why not in the database worker
|
||||
|
||||
In the browser: a **dedicated worker owned by the tab**, separate from the
|
||||
SharedWorker that owns the database.
|
||||
|
||||
- Plugin work is slow by design, and the database worker answers every tab's
|
||||
commands synchronously. A large import in there would stall every other tab's
|
||||
reads.
|
||||
- A plugin that never returns can be ended with `terminate()`. You cannot do
|
||||
that to the worker holding the database.
|
||||
- The capabilities plugins actually ask for — a prompt, a toast, the active
|
||||
request — belong to a tab, not to a database. Routing through the tab is the
|
||||
shorter path, not a detour.
|
||||
|
||||
The cost is that `ctx.store` goes worker → tab → database worker. It is a message
|
||||
either way, and this is the direction where a stuck plugin costs nothing.
|
||||
|
||||
Template rendering is the one flow that runs backwards: rendering happens in the
|
||||
engine, in the database worker, but the functions it calls live here. So the
|
||||
engine is handed a callback that asks the tab, which asks the sandbox. See
|
||||
`templateBridge` in `packages/platform/src/web/worker.ts`.
|
||||
|
||||
## Plugins versus scripts
|
||||
|
||||
The shell is **not plugin-shaped underneath**. `load` takes source; `dispatch`
|
||||
takes an event. What a module *is* — a plugin today, a workspace script later —
|
||||
is decided by the payloads the host sends, not by the runtime.
|
||||
|
||||
That matters for one reason. A plugin is installed, so someone consented to it,
|
||||
and a plugin may one day escalate to a full Node runtime by asking. **A script
|
||||
arrives inside a workspace — as data, through an import, a git sync, a shared
|
||||
repository — with no consent moment at all.** So scripts get this sandbox and
|
||||
only this sandbox, forever, regardless of feature pressure. Any capability added
|
||||
below must be evaluated against the script case, which is the stricter one:
|
||||
"would I want this to run because someone opened a workspace a stranger sent
|
||||
them?"
|
||||
|
||||
Expected differences when scripts arrive, none of them built yet:
|
||||
|
||||
- A different payload set (`run_script_request` and friends) — same envelope.
|
||||
- A tighter host-call allowlist. A script should probably not reach `ctx.store`
|
||||
at all, and certainly not another plugin's namespace.
|
||||
- A much shorter watchdog. A pre-request script that runs for a minute is broken;
|
||||
an importer that does is working.
|
||||
|
||||
## Performance
|
||||
|
||||
QuickJS is an interpreter with no JIT. Measured on GitHub's 12.3 MB OpenAPI
|
||||
description (1220 requests imported, **identical output** in both engines):
|
||||
|
||||
| | First run | Best of 6 |
|
||||
|---|---|---|
|
||||
| Node (V8) | 304 ms | 164 ms |
|
||||
| QuickJS sandbox | 2503 ms | 2017 ms |
|
||||
|
||||
That is **8x on the first run** and about **12x once V8 has compiled** — well
|
||||
inside the 10–50x folklore, and the first-run number is the one a user waits for
|
||||
because an import happens once. Reproduce with:
|
||||
|
||||
```bash
|
||||
node packages/plugin-sandbox/bench/import.mjs <spec.json> 6
|
||||
```
|
||||
|
||||
**Conclusion: importers stay in the sandbox.** 2.5 s in a worker, behind a
|
||||
progress state, for the largest public API description that exists, is a fine
|
||||
trade for one runtime everywhere. Revisit if a real document is measured
|
||||
materially worse — the escape hatch is a host builtin for the hot path, not a
|
||||
second runtime.
|
||||
|
||||
Boot cost is small: about 80–140 ms to instantiate the wasm and load a plugin,
|
||||
paid once and lazily, so a session that never touches a plugin never pays it.
|
||||
The wasm is 529 KB, next to the 4.3 MB SQLite one.
|
||||
@@ -0,0 +1,140 @@
|
||||
/**
|
||||
* How much slower is an importer inside the sandbox?
|
||||
*
|
||||
* This is the number the tiered-runtime decision rests on. Template functions
|
||||
* and auth signing are small enough that engine speed cannot matter; importing
|
||||
* is not. Yaak's OpenAPI importer is first-party JavaScript, not Rust, so a
|
||||
* large specification is parsed and walked by whatever engine the runtime uses
|
||||
* — QuickJS in a browser tab, V8 on the desktop today. If the gap is large
|
||||
* enough to be felt on a real document, importers need a different path before
|
||||
* the corpus is ported.
|
||||
*
|
||||
* Usage:
|
||||
* node packages/plugin-sandbox/bench/import.mjs <spec.json> [iterations]
|
||||
*/
|
||||
|
||||
import { build } from "esbuild";
|
||||
import { mkdirSync, readFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { bundlePlugin } from "../../../scripts/bundle-sandbox-plugins.mjs";
|
||||
|
||||
const root = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
||||
const PLUGIN = "importer-openapi";
|
||||
|
||||
const specPath = process.argv[2];
|
||||
const iterations = Number(process.argv[3] ?? 3);
|
||||
if (specPath == null) {
|
||||
console.error("usage: node bench/import.mjs <spec.json> [iterations]");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const spec = readFileSync(specPath, "utf8");
|
||||
console.log(`Spec: ${specPath} (${(spec.length / 1024 / 1024).toFixed(1)} MB)`);
|
||||
console.log(`Iterations: ${iterations}\n`);
|
||||
|
||||
/** The sandbox host, bundled for Node so this script can drive it directly. */
|
||||
async function loadHost() {
|
||||
// Inside node_modules so the emitted bundle's own imports of the QuickJS
|
||||
// variant resolve the way any other module's would.
|
||||
const outDir = join(root, "node_modules", ".cache", "yaak-plugin-sandbox");
|
||||
mkdirSync(outDir, { recursive: true });
|
||||
const outfile = join(outDir, "host.mjs");
|
||||
await build({
|
||||
entryPoints: [join(root, "packages/plugin-sandbox/src/host/sandbox.ts")],
|
||||
bundle: true,
|
||||
format: "esm",
|
||||
platform: "node",
|
||||
target: "node22",
|
||||
outfile,
|
||||
// Resolved from the repo at run time, so the wasm variant is the real one.
|
||||
external: ["@jitl/*", "quickjs-emscripten-core"],
|
||||
});
|
||||
return import(pathToFileURL(outfile).href);
|
||||
}
|
||||
|
||||
const ctxStub = { id: "bench", label: null, workspaceId: "wk_bench" };
|
||||
|
||||
function stats(times) {
|
||||
const sorted = [...times].sort((a, b) => a - b);
|
||||
const mean = times.reduce((a, b) => a + b, 0) / times.length;
|
||||
return { min: sorted[0], median: sorted[Math.floor(sorted.length / 2)], mean };
|
||||
}
|
||||
|
||||
function report(label, times, resourceCount) {
|
||||
const { min, median } = stats(times);
|
||||
console.log(
|
||||
`${label.padEnd(20)} first ${times[0].toFixed(0).padStart(5)} ms ` +
|
||||
`best ${min.toFixed(0).padStart(5)} ms ` +
|
||||
`median ${median.toFixed(0).padStart(5)} ms (${resourceCount} requests)`,
|
||||
);
|
||||
// Every run, because the spread is the point: V8 compiles this workload
|
||||
// across the first few passes and QuickJS, which does not compile at all,
|
||||
// does not. Quoting one ratio would pick a winner by choosing when to look.
|
||||
console.log(`${" ".repeat(20)} runs: ${times.map((t) => t.toFixed(0)).join(", ")} ms`);
|
||||
return { first: times[0], best: min };
|
||||
}
|
||||
|
||||
/* --------------------------------- Node ---------------------------------- */
|
||||
|
||||
const nodeTimes = [];
|
||||
let nodeCount = 0;
|
||||
{
|
||||
const { createRequire } = await import("node:module");
|
||||
const require = createRequire(join(root, "package.json"));
|
||||
const mod = require(join(root, "plugins", PLUGIN, "build", "index.js"));
|
||||
const plugin = mod.plugin ?? mod.default;
|
||||
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
const started = performance.now();
|
||||
const result = await plugin.importer.onImport(ctxStub, { text: spec });
|
||||
nodeTimes.push(performance.now() - started);
|
||||
nodeCount = result?.resources?.httpRequests?.length ?? 0;
|
||||
}
|
||||
}
|
||||
const node = report("Node (V8)", nodeTimes, nodeCount);
|
||||
|
||||
/* -------------------------------- QuickJS -------------------------------- */
|
||||
|
||||
const quickTimes = [];
|
||||
let quickCount = 0;
|
||||
{
|
||||
const { PluginSandboxHost } = await loadHost();
|
||||
const source = await bundlePlugin(PLUGIN);
|
||||
|
||||
const host = new PluginSandboxHost(
|
||||
async () => JSON.stringify({ type: "empty_response" }),
|
||||
(log) => console.error(`[${log.level}] ${log.message}`),
|
||||
);
|
||||
|
||||
const loadStarted = performance.now();
|
||||
await host.load(PLUGIN, source);
|
||||
console.log(`(sandbox boot + load: ${(performance.now() - loadStarted).toFixed(0)} ms)\n`);
|
||||
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
const started = performance.now();
|
||||
const reply = JSON.parse(
|
||||
await host.dispatch(
|
||||
PLUGIN,
|
||||
JSON.stringify({ context: ctxStub, payload: { type: "import_request", content: spec } }),
|
||||
),
|
||||
);
|
||||
quickTimes.push(performance.now() - started);
|
||||
if (reply.type === "error_response") throw new Error(reply.error);
|
||||
quickCount = reply.resources?.httpRequests?.length ?? 0;
|
||||
}
|
||||
host.dispose();
|
||||
}
|
||||
const quick = report("QuickJS (sandbox)", quickTimes, quickCount);
|
||||
|
||||
console.log(
|
||||
`\nFirst run (what a user waits for): ${(quick.first / 1000).toFixed(1)}s in the sandbox ` +
|
||||
`vs ${(node.first / 1000).toFixed(1)}s in Node — ${(quick.first / node.first).toFixed(1)}x.`,
|
||||
);
|
||||
console.log(
|
||||
`Best run (both warm): ${(quick.best / node.best).toFixed(1)}x, which is the ceiling once V8 has compiled.`,
|
||||
);
|
||||
if (nodeCount !== quickCount) {
|
||||
console.log(`WARNING: request counts differ (${nodeCount} vs ${quickCount}) — not the same work.`);
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* Bundle the guest shell into a string the host can evaluate.
|
||||
*
|
||||
* The shell runs inside QuickJS, which has no module loader and no filesystem,
|
||||
* so it has to arrive as source text. Emitting it as a `.ts` module rather than
|
||||
* a `.js` asset is what lets every consumer — Vite for the browser build, plain
|
||||
* Node for the benchmarks — get at it the same way, with no loader plugin and
|
||||
* no `?raw` import that only one bundler understands.
|
||||
*
|
||||
* The output is committed, like the wasm packages are, so a checkout builds
|
||||
* without this step having run.
|
||||
*/
|
||||
|
||||
import { build } from "esbuild";
|
||||
import { mkdirSync, writeFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const outDir = join(here, "src", "generated");
|
||||
|
||||
const result = await build({
|
||||
entryPoints: [join(here, "src", "guest", "index.ts")],
|
||||
bundle: true,
|
||||
write: false,
|
||||
// A script, not a module: the host evaluates it with `evalCode`, and it
|
||||
// announces itself by assigning `globalThis.__yaak_guest`.
|
||||
format: "iife",
|
||||
// Nothing here may reach for a Node built-in, and "browser" is the closest
|
||||
// description of a target with globals and no filesystem. QuickJS itself has
|
||||
// fewer globals than any browser, which is what `guest/globals.ts` is for.
|
||||
platform: "browser",
|
||||
// QuickJS is ES2023-complete, so nothing needs downleveling. Keeping the
|
||||
// source as written also keeps stack traces from the guest readable.
|
||||
target: "es2022",
|
||||
minify: false,
|
||||
legalComments: "none",
|
||||
});
|
||||
|
||||
const source = result.outputFiles[0].text;
|
||||
|
||||
mkdirSync(outDir, { recursive: true });
|
||||
writeFileSync(
|
||||
join(outDir, "guest.ts"),
|
||||
[
|
||||
"// Generated by build-guest.mjs. Do not edit.",
|
||||
"//",
|
||||
"// The runtime shell, as source text, for evaluation inside QuickJS.",
|
||||
"// Regenerate with `npm run build --workspace @yaakapp-internal/plugin-sandbox`.",
|
||||
"",
|
||||
`export const GUEST_SOURCE = ${JSON.stringify(source)};`,
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
|
||||
console.log(`Bundled guest shell: ${(source.length / 1024).toFixed(1)} KB`);
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"name": "@yaakapp-internal/plugin-sandbox",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"main": "src/index.ts",
|
||||
"scripts": {
|
||||
"bootstrap": "npm run build",
|
||||
"build": "node build-guest.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@jitl/quickjs-ng-wasmfile-release-sync": "^0.32.0",
|
||||
"quickjs-emscripten-core": "^0.32.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"esbuild": "^0.28.0"
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,370 @@
|
||||
/**
|
||||
* `ctx`, as a plugin sees it, built entirely out of one call to the host.
|
||||
*
|
||||
* Every method here serializes a request payload, hands it out of the sandbox,
|
||||
* and awaits a reply payload. That is the whole capability surface: the sandbox
|
||||
* has no socket, no clock it owns, no storage and no DOM, so anything a plugin
|
||||
* does to the world is a message the host chose to answer. The payload shapes
|
||||
* are the ones in `crates/yaak-plugins/src/events.rs`, unchanged, so a plugin
|
||||
* written for the Node runtime runs here without knowing which host it has.
|
||||
*/
|
||||
|
||||
import type {
|
||||
CallPromptFormDynamicArgs,
|
||||
Context,
|
||||
DynamicPromptFormArg,
|
||||
} from "@yaakapp/api";
|
||||
import {
|
||||
applyDynamicFormInput,
|
||||
stripDynamicCallbacks,
|
||||
} from "@yaakapp-internal/lib/pluginForms";
|
||||
import { createResponseBody, decodeBase64Chunk } from "@yaakapp-internal/lib/responseBody";
|
||||
import { applyFormInputDefaults } from "@yaakapp-internal/lib/templateFunction";
|
||||
import type {
|
||||
DeleteKeyValueResponse,
|
||||
DeleteModelResponse,
|
||||
FindHttpResponsesResponse,
|
||||
Folder,
|
||||
FormInput,
|
||||
GetCookieValueRequest,
|
||||
GetCookieValueResponse,
|
||||
GetHttpRequestByIdResponse,
|
||||
GetHttpResponseBodyInfoResponse,
|
||||
GetKeyValueResponse,
|
||||
HttpRequest,
|
||||
HttpResponse,
|
||||
InternalEventPayload,
|
||||
ListCookieNamesResponse,
|
||||
ListFoldersResponse,
|
||||
ListHttpRequestsRequest,
|
||||
ListHttpRequestsResponse,
|
||||
ListOpenWorkspacesResponse,
|
||||
PluginContext,
|
||||
PromptFormResponse,
|
||||
PromptTextResponse,
|
||||
ReadHttpResponseBodyChunkResponse,
|
||||
RenderGrpcRequestResponse,
|
||||
RenderHttpRequestResponse,
|
||||
SendHttpRequestResponse,
|
||||
TemplateRenderRequest,
|
||||
TemplateRenderResponse,
|
||||
UpsertModelResponse,
|
||||
WindowInfoResponse,
|
||||
} from "@yaakapp-internal/plugins";
|
||||
|
||||
/** What the host installs: one request out, one reply back. */
|
||||
export type HostCall = (
|
||||
context: PluginContext,
|
||||
payload: InternalEventPayload,
|
||||
) => Promise<Record<string, unknown>>;
|
||||
|
||||
/**
|
||||
* A response as a plugin should see it.
|
||||
*
|
||||
* `bodyPath` names a file on a host's disk. There is no disk here and there is
|
||||
* none in a browser, and plugins address bodies by response id, so it is
|
||||
* dropped rather than left for one to grow a dependency on.
|
||||
*/
|
||||
function forPlugin(httpResponse: HttpResponse): HttpResponse {
|
||||
const { bodyPath: _bodyPath, ...rest } = httpResponse as HttpResponse & {
|
||||
bodyPath?: string | null;
|
||||
};
|
||||
return rest;
|
||||
}
|
||||
|
||||
export function newContext(call: HostCall, context: PluginContext): Context {
|
||||
const send = <T>(payload: InternalEventPayload): Promise<T> =>
|
||||
call(context, payload) as Promise<T>;
|
||||
|
||||
/** Read a body the host has stored, a chunk at a time, following it if it is still arriving. */
|
||||
const storedBody = async (responseId: string) => {
|
||||
const bodyInfo = () =>
|
||||
send<GetHttpResponseBodyInfoResponse>({
|
||||
type: "get_http_response_body_info_request",
|
||||
responseId,
|
||||
});
|
||||
const info = await bodyInfo();
|
||||
|
||||
return createResponseBody(
|
||||
{
|
||||
responseId,
|
||||
contentLength: info.contentLength,
|
||||
contentType: info.contentType ?? null,
|
||||
complete: info.complete,
|
||||
},
|
||||
async (offset, length) => {
|
||||
const chunk = await send<ReadHttpResponseBodyChunkResponse>({
|
||||
type: "read_http_response_body_chunk_request",
|
||||
responseId,
|
||||
offset,
|
||||
length,
|
||||
});
|
||||
return decodeBase64Chunk(chunk.data);
|
||||
},
|
||||
{ refresh: bodyInfo },
|
||||
);
|
||||
};
|
||||
|
||||
const windowInfo = async () => {
|
||||
if (context.label == null) {
|
||||
throw new Error("Can't get window context without an active window");
|
||||
}
|
||||
return send<WindowInfoResponse>({ type: "window_info_request", label: context.label });
|
||||
};
|
||||
|
||||
const ctx: Context = {
|
||||
clipboard: {
|
||||
copyText: async (text) => {
|
||||
await send({ type: "copy_text_request", text });
|
||||
},
|
||||
},
|
||||
toast: {
|
||||
show: async (args) => {
|
||||
await send({
|
||||
type: "show_toast_request",
|
||||
// Defaulted here because null and undefined both become None in Rust.
|
||||
timeout: args.timeout === undefined ? 5000 : args.timeout,
|
||||
...args,
|
||||
});
|
||||
},
|
||||
},
|
||||
window: {
|
||||
requestId: async () => (await windowInfo()).requestId,
|
||||
workspaceId: async () => (await windowInfo()).workspaceId,
|
||||
environmentId: async () => (await windowInfo()).environmentId,
|
||||
openUrl: async () => {
|
||||
// A window is the host's to open, and the browser host has one tab. A
|
||||
// plugin asking is told so rather than handed a handle that does
|
||||
// nothing when it calls `close()`.
|
||||
throw new Error("ctx.window.openUrl is not available in the sandbox runtime");
|
||||
},
|
||||
openExternalUrl: async (url) => {
|
||||
await send({ type: "open_external_url_request", url });
|
||||
},
|
||||
},
|
||||
prompt: {
|
||||
text: async (args) => {
|
||||
const reply = await send<PromptTextResponse>({ type: "prompt_text_request", ...args });
|
||||
return reply.value;
|
||||
},
|
||||
form: async (args) => {
|
||||
// The inputs a plugin declares may compute themselves from the values
|
||||
// entered so far. The host draws a static form, so they are resolved
|
||||
// against the defaults before it is drawn and the callbacks stripped
|
||||
// — a function cannot cross the boundary, and one left in would
|
||||
// serialize to nothing and take its input's shape with it.
|
||||
const defaults = applyFormInputDefaults(args.inputs, {});
|
||||
const callArgs: CallPromptFormDynamicArgs = { values: defaults };
|
||||
const resolved = await applyDynamicFormInput(
|
||||
ctx,
|
||||
args.inputs as DynamicPromptFormArg[],
|
||||
callArgs,
|
||||
);
|
||||
const reply = await send<PromptFormResponse>({
|
||||
type: "prompt_form_request",
|
||||
...args,
|
||||
inputs: stripDynamicCallbacks(resolved) as FormInput[],
|
||||
});
|
||||
return reply.values;
|
||||
},
|
||||
},
|
||||
httpResponse: {
|
||||
find: async (args) => {
|
||||
const { httpResponses } = await send<FindHttpResponsesResponse>({
|
||||
type: "find_http_responses_request",
|
||||
...args,
|
||||
});
|
||||
return httpResponses.map(forPlugin);
|
||||
},
|
||||
body: ({ responseId }) => storedBody(responseId),
|
||||
},
|
||||
grpcRequest: {
|
||||
render: async (args) => {
|
||||
const { grpcRequest } = await send<RenderGrpcRequestResponse>({
|
||||
type: "render_grpc_request_request",
|
||||
...args,
|
||||
});
|
||||
return grpcRequest;
|
||||
},
|
||||
},
|
||||
httpRequest: {
|
||||
getById: async (args) => {
|
||||
const { httpRequest } = await send<GetHttpRequestByIdResponse>({
|
||||
type: "get_http_request_by_id_request",
|
||||
...args,
|
||||
});
|
||||
return httpRequest;
|
||||
},
|
||||
send: async (args) => {
|
||||
const { httpResponse, body } = await send<SendHttpRequestResponse>({
|
||||
type: "send_http_request_request",
|
||||
...args,
|
||||
});
|
||||
|
||||
// A send with no request behind it saves nothing, so the reply carries
|
||||
// the only copy of its body. A saved one is read back from the host
|
||||
// like any other. Callers get the same thing either way.
|
||||
if (body == null) {
|
||||
return {
|
||||
httpResponse: forPlugin(httpResponse),
|
||||
body: await storedBody(httpResponse.id),
|
||||
};
|
||||
}
|
||||
|
||||
const bytes = decodeBase64Chunk(body);
|
||||
return {
|
||||
httpResponse: forPlugin(httpResponse),
|
||||
body: createResponseBody(
|
||||
{
|
||||
responseId: httpResponse.id,
|
||||
contentLength: bytes.byteLength,
|
||||
contentType:
|
||||
httpResponse.headers.find((h) => h.name.toLowerCase() === "content-type")?.value ??
|
||||
null,
|
||||
// The host waited for the whole send before replying.
|
||||
complete: true,
|
||||
},
|
||||
async (offset, length) => bytes.slice(offset, offset + length),
|
||||
),
|
||||
};
|
||||
},
|
||||
render: async (args) => {
|
||||
const { httpRequest } = await send<RenderHttpRequestResponse>({
|
||||
type: "render_http_request_request",
|
||||
...args,
|
||||
});
|
||||
return httpRequest;
|
||||
},
|
||||
list: async (args?: { folderId?: string }) => {
|
||||
const payload: InternalEventPayload = {
|
||||
type: "list_http_requests_request",
|
||||
folderId: args?.folderId,
|
||||
} satisfies ListHttpRequestsRequest & { type: "list_http_requests_request" };
|
||||
const { httpRequests } = await send<ListHttpRequestsResponse>(payload);
|
||||
return httpRequests;
|
||||
},
|
||||
create: async (args) => {
|
||||
const response = await send<UpsertModelResponse>({
|
||||
type: "upsert_model_request",
|
||||
model: { name: "", method: "GET", ...args, id: "", model: "http_request" },
|
||||
} as InternalEventPayload);
|
||||
return response.model as HttpRequest;
|
||||
},
|
||||
update: async (args) => {
|
||||
const response = await send<UpsertModelResponse>({
|
||||
type: "upsert_model_request",
|
||||
model: { model: "http_request", ...args },
|
||||
} as InternalEventPayload);
|
||||
return response.model as HttpRequest;
|
||||
},
|
||||
delete: async (args) => {
|
||||
const response = await send<DeleteModelResponse>({
|
||||
type: "delete_model_request",
|
||||
model: "http_request",
|
||||
id: args.id,
|
||||
} as InternalEventPayload);
|
||||
return response.model as HttpRequest;
|
||||
},
|
||||
},
|
||||
folder: {
|
||||
list: async () => {
|
||||
const { folders } = await send<ListFoldersResponse>({ type: "list_folders_request" });
|
||||
return folders;
|
||||
},
|
||||
getById: async (args: { id: string }) => {
|
||||
const { folders } = await send<ListFoldersResponse>({ type: "list_folders_request" });
|
||||
return folders.find((f) => f.id === args.id) ?? null;
|
||||
},
|
||||
create: async ({ name, ...args }) => {
|
||||
const response = await send<UpsertModelResponse>({
|
||||
type: "upsert_model_request",
|
||||
model: { ...args, name: name ?? "", id: "", model: "folder" },
|
||||
} as InternalEventPayload);
|
||||
return response.model as Folder;
|
||||
},
|
||||
update: async (args) => {
|
||||
const response = await send<UpsertModelResponse>({
|
||||
type: "upsert_model_request",
|
||||
model: { model: "folder", ...args },
|
||||
} as InternalEventPayload);
|
||||
return response.model as Folder;
|
||||
},
|
||||
delete: async (args: { id: string }) => {
|
||||
const response = await send<DeleteModelResponse>({
|
||||
type: "delete_model_request",
|
||||
model: "folder",
|
||||
id: args.id,
|
||||
} as InternalEventPayload);
|
||||
return response.model as Folder;
|
||||
},
|
||||
},
|
||||
cookies: {
|
||||
getValue: async (args: GetCookieValueRequest) => {
|
||||
const { value } = await send<GetCookieValueResponse>({
|
||||
type: "get_cookie_value_request",
|
||||
...args,
|
||||
});
|
||||
return value;
|
||||
},
|
||||
listNames: async () => {
|
||||
const { names } = await send<ListCookieNamesResponse>({ type: "list_cookie_names_request" });
|
||||
return names;
|
||||
},
|
||||
},
|
||||
templates: {
|
||||
render: async (args: TemplateRenderRequest) => {
|
||||
const result = await send<TemplateRenderResponse>({
|
||||
type: "template_render_request",
|
||||
...args,
|
||||
});
|
||||
// oxlint-disable-next-line no-explicit-any -- the caller knows its own shape
|
||||
return result.data as any;
|
||||
},
|
||||
},
|
||||
store: {
|
||||
get: async <T>(key: string) => {
|
||||
const result = await send<GetKeyValueResponse>({ type: "get_key_value_request", key });
|
||||
return result.value ? (JSON.parse(result.value) as T) : undefined;
|
||||
},
|
||||
set: async <T>(key: string, value: T) => {
|
||||
await send<GetKeyValueResponse>({
|
||||
type: "set_key_value_request",
|
||||
key,
|
||||
value: JSON.stringify(value),
|
||||
});
|
||||
},
|
||||
delete: async (key: string) => {
|
||||
const result = await send<DeleteKeyValueResponse>({
|
||||
type: "delete_key_value_request",
|
||||
key,
|
||||
});
|
||||
return result.deleted;
|
||||
},
|
||||
},
|
||||
plugin: {
|
||||
reload: () => {
|
||||
void send({ type: "reload_response", silent: true });
|
||||
},
|
||||
},
|
||||
workspace: {
|
||||
list: async () => {
|
||||
const response = await send<ListOpenWorkspacesResponse>({
|
||||
type: "list_open_workspaces_request",
|
||||
});
|
||||
return response.workspaces.map((w) => {
|
||||
type WorkspaceInfoInternal = typeof w & { label?: string };
|
||||
return {
|
||||
id: w.id,
|
||||
name: w.name,
|
||||
// Kept for routing, hidden from plugin authors.
|
||||
_label: (w as WorkspaceInfoInternal).label as string,
|
||||
};
|
||||
});
|
||||
},
|
||||
withContext: (handle: { id: string; name: string; _label?: string }) =>
|
||||
newContext(call, { ...context, label: handle._label || null, workspaceId: handle.id }),
|
||||
},
|
||||
};
|
||||
|
||||
return ctx;
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
/**
|
||||
* The globals that exist inside the sandbox.
|
||||
*
|
||||
* QuickJS is the language and nothing else: it has `Promise`, `BigInt` and the
|
||||
* ES2024 built-ins, and no `console`, no `setTimeout`, no `TextEncoder`. The
|
||||
* platform globals a browser or Node would supply are not there because there
|
||||
* is no platform — which is the point. What a plugin can reach is what this
|
||||
* file installs, and every one of these has to exist identically on the Rust
|
||||
* host too, so the list is kept short and boring on purpose.
|
||||
*
|
||||
* Two of them are implemented here in pure JavaScript rather than bridged to
|
||||
* the host: the text codecs are twenty lines and a bridge would cost a copy
|
||||
* each way for no gain. Timers cannot be — the sandbox has no event loop of its
|
||||
* own — so those are the host's.
|
||||
*/
|
||||
|
||||
declare const __yaak_log: (level: string, message: string) => void;
|
||||
declare const __yaak_timer_start: (id: number, ms: number) => void;
|
||||
declare const __yaak_timer_cancel: (id: number) => void;
|
||||
|
||||
/* -------------------------------- console -------------------------------- */
|
||||
|
||||
/**
|
||||
* Arguments as a line of text, formatted here rather than at the host.
|
||||
*
|
||||
* Only strings cross the boundary, so a plugin logging an object gets it
|
||||
* serialized inside the sandbox, where its own prototypes still exist and a
|
||||
* cycle is this function's problem rather than the host's.
|
||||
*/
|
||||
function formatArgs(args: unknown[]): string {
|
||||
return args
|
||||
.map((arg) => {
|
||||
if (typeof arg === "string") return arg;
|
||||
if (arg instanceof Error) return arg.stack ?? `${arg.name}: ${arg.message}`;
|
||||
try {
|
||||
return JSON.stringify(arg, replacer()) ?? String(arg);
|
||||
} catch {
|
||||
return String(arg);
|
||||
}
|
||||
})
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
function replacer(): (key: string, value: unknown) => unknown {
|
||||
const seen = new WeakSet<object>();
|
||||
return (_key, value) => {
|
||||
if (typeof value === "bigint") return `${value}n`;
|
||||
if (typeof value === "function") return `[Function ${value.name || "anonymous"}]`;
|
||||
if (typeof value === "object" && value !== null) {
|
||||
if (seen.has(value)) return "[Circular]";
|
||||
seen.add(value);
|
||||
}
|
||||
return value;
|
||||
};
|
||||
}
|
||||
|
||||
function installConsole(): void {
|
||||
const log = (level: string) => (...args: unknown[]) => __yaak_log(level, formatArgs(args));
|
||||
(globalThis as Record<string, unknown>).console = {
|
||||
log: log("log"),
|
||||
info: log("info"),
|
||||
warn: log("warn"),
|
||||
error: log("error"),
|
||||
debug: log("debug"),
|
||||
trace: log("debug"),
|
||||
};
|
||||
}
|
||||
|
||||
/* --------------------------------- timers -------------------------------- */
|
||||
|
||||
/**
|
||||
* Timers, owned by the host.
|
||||
*
|
||||
* QuickJS has no clock to wake on: `executePendingJobs` drains microtasks and
|
||||
* returns, so a `setTimeout` implemented in here would either never fire or
|
||||
* spin. The host holds the real timer and calls back in, which also means a
|
||||
* sandbox torn down mid-wait takes its pending timers with it.
|
||||
*/
|
||||
const timerCallbacks = new Map<number, () => void>();
|
||||
let nextTimerId = 1;
|
||||
|
||||
function installTimers(): void {
|
||||
const g = globalThis as Record<string, unknown>;
|
||||
|
||||
g.setTimeout = (callback: (...a: unknown[]) => void, ms?: number, ...args: unknown[]) => {
|
||||
const id = nextTimerId++;
|
||||
timerCallbacks.set(id, () => callback(...args));
|
||||
__yaak_timer_start(id, Math.max(0, Number(ms) || 0));
|
||||
return id;
|
||||
};
|
||||
|
||||
g.clearTimeout = (id: number) => {
|
||||
if (!timerCallbacks.delete(id)) return;
|
||||
__yaak_timer_cancel(id);
|
||||
};
|
||||
|
||||
// Same contract, and deliberately not repeating: an interval is a timer that
|
||||
// rearms, and nothing in a plugin should be polling anyway. A plugin that
|
||||
// wants one can build it from `setTimeout`, visibly.
|
||||
g.setInterval = undefined;
|
||||
g.clearInterval = undefined;
|
||||
}
|
||||
|
||||
/** Called by the host when a timer it is holding comes due. */
|
||||
function fireTimer(id: number): void {
|
||||
const callback = timerCallbacks.get(id);
|
||||
timerCallbacks.delete(id);
|
||||
callback?.();
|
||||
}
|
||||
|
||||
/* ------------------------------- text codecs ------------------------------ */
|
||||
|
||||
class SandboxTextEncoder {
|
||||
readonly encoding = "utf-8";
|
||||
|
||||
encode(input = ""): Uint8Array {
|
||||
const out: number[] = [];
|
||||
for (let i = 0; i < input.length; i++) {
|
||||
let code = input.charCodeAt(i);
|
||||
// A surrogate pair is one code point; a lone surrogate becomes U+FFFD,
|
||||
// which is what the standard encoder does rather than erroring.
|
||||
if (code >= 0xd800 && code <= 0xdbff) {
|
||||
const next = input.charCodeAt(i + 1);
|
||||
if (next >= 0xdc00 && next <= 0xdfff) {
|
||||
code = (code - 0xd800) * 0x400 + (next - 0xdc00) + 0x10000;
|
||||
i++;
|
||||
} else {
|
||||
code = 0xfffd;
|
||||
}
|
||||
} else if (code >= 0xdc00 && code <= 0xdfff) {
|
||||
code = 0xfffd;
|
||||
}
|
||||
|
||||
if (code < 0x80) out.push(code);
|
||||
else if (code < 0x800) out.push(0xc0 | (code >> 6), 0x80 | (code & 0x3f));
|
||||
else if (code < 0x10000)
|
||||
out.push(0xe0 | (code >> 12), 0x80 | ((code >> 6) & 0x3f), 0x80 | (code & 0x3f));
|
||||
else
|
||||
out.push(
|
||||
0xf0 | (code >> 18),
|
||||
0x80 | ((code >> 12) & 0x3f),
|
||||
0x80 | ((code >> 6) & 0x3f),
|
||||
0x80 | (code & 0x3f),
|
||||
);
|
||||
}
|
||||
return new Uint8Array(out);
|
||||
}
|
||||
}
|
||||
|
||||
class SandboxTextDecoder {
|
||||
readonly encoding = "utf-8";
|
||||
|
||||
decode(input?: ArrayBuffer | ArrayBufferView): string {
|
||||
if (input == null) return "";
|
||||
const bytes =
|
||||
input instanceof Uint8Array
|
||||
? input
|
||||
: ArrayBuffer.isView(input)
|
||||
? new Uint8Array(input.buffer, input.byteOffset, input.byteLength)
|
||||
: new Uint8Array(input);
|
||||
|
||||
let out = "";
|
||||
for (let i = 0; i < bytes.length; ) {
|
||||
const byte = bytes[i]!;
|
||||
let code: number;
|
||||
let size: number;
|
||||
if (byte < 0x80) {
|
||||
code = byte;
|
||||
size = 1;
|
||||
} else if ((byte & 0xe0) === 0xc0) {
|
||||
code = byte & 0x1f;
|
||||
size = 2;
|
||||
} else if ((byte & 0xf0) === 0xe0) {
|
||||
code = byte & 0x0f;
|
||||
size = 3;
|
||||
} else if ((byte & 0xf8) === 0xf0) {
|
||||
code = byte & 0x07;
|
||||
size = 4;
|
||||
} else {
|
||||
out += "�";
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (i + size > bytes.length) {
|
||||
out += "�";
|
||||
break;
|
||||
}
|
||||
for (let k = 1; k < size; k++) {
|
||||
const cont = bytes[i + k]!;
|
||||
if ((cont & 0xc0) !== 0x80) {
|
||||
code = -1;
|
||||
break;
|
||||
}
|
||||
code = (code << 6) | (cont & 0x3f);
|
||||
}
|
||||
i += size;
|
||||
|
||||
if (code < 0 || code > 0x10ffff || (code >= 0xd800 && code <= 0xdfff)) out += "�";
|
||||
else if (code < 0x10000) out += String.fromCharCode(code);
|
||||
else {
|
||||
const c = code - 0x10000;
|
||||
out += String.fromCharCode(0xd800 + (c >> 10), 0xdc00 + (c & 0x3ff));
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
}
|
||||
|
||||
function installTextCodecs(): void {
|
||||
const g = globalThis as Record<string, unknown>;
|
||||
g.TextEncoder = SandboxTextEncoder;
|
||||
g.TextDecoder = SandboxTextDecoder;
|
||||
}
|
||||
|
||||
/* ------------------------------ base64 helpers ---------------------------- */
|
||||
|
||||
const B64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
|
||||
function installBase64(): void {
|
||||
const g = globalThis as Record<string, unknown>;
|
||||
|
||||
// Latin-1 in, base64 out — the same narrow contract the browser's have, so a
|
||||
// plugin that reaches for them behaves the same here as it does there.
|
||||
g.btoa = (input: string): string => {
|
||||
let out = "";
|
||||
for (let i = 0; i < input.length; i += 3) {
|
||||
const a = input.charCodeAt(i);
|
||||
const b = input.charCodeAt(i + 1);
|
||||
const c = input.charCodeAt(i + 2);
|
||||
if (a > 0xff || b > 0xff || c > 0xff) {
|
||||
throw new Error("btoa: string contains characters outside of the Latin1 range");
|
||||
}
|
||||
const chunk = (a << 16) | ((Number.isNaN(b) ? 0 : b) << 8) | (Number.isNaN(c) ? 0 : c);
|
||||
out += B64[(chunk >> 18) & 63]! + B64[(chunk >> 12) & 63]!;
|
||||
out += Number.isNaN(b) ? "=" : B64[(chunk >> 6) & 63]!;
|
||||
out += Number.isNaN(c) ? "=" : B64[chunk & 63]!;
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
g.atob = (input: string): string => {
|
||||
const clean = input.replace(/[\t\n\f\r ]/g, "").replace(/=+$/, "");
|
||||
let out = "";
|
||||
let bits = 0;
|
||||
let acc = 0;
|
||||
for (const ch of clean) {
|
||||
const value = B64.indexOf(ch);
|
||||
if (value < 0) throw new Error("atob: string contains invalid characters");
|
||||
acc = (acc << 6) | value;
|
||||
bits += 6;
|
||||
if (bits >= 8) {
|
||||
bits -= 8;
|
||||
out += String.fromCharCode((acc >> bits) & 0xff);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
};
|
||||
}
|
||||
|
||||
export function installGlobals(): { fireTimer: (id: number) => void } {
|
||||
installConsole();
|
||||
installTimers();
|
||||
installTextCodecs();
|
||||
installBase64();
|
||||
return { fireTimer };
|
||||
}
|
||||
@@ -0,0 +1,355 @@
|
||||
/**
|
||||
* The runtime shell, as it exists inside the sandbox.
|
||||
*
|
||||
* This is the whole of what QuickJS evaluates before any untrusted code does:
|
||||
* it installs the globals, loads one module, and answers events against it.
|
||||
* The Node runtime's `PluginInstance` does the same job on the other side of a
|
||||
* WebSocket; the difference is that this one has no filesystem to load from and
|
||||
* no host objects to reach for, so the module arrives as source text and every
|
||||
* capability arrives as a reply.
|
||||
*
|
||||
* It is deliberately not plugin-shaped underneath. `load` takes source and
|
||||
* `dispatch` takes an event: what a module *is* — a plugin today, a workspace
|
||||
* script later — is decided by the payloads the host sends, not by this file.
|
||||
* Scripts are the reason that matters. A plugin is installed, so someone
|
||||
* consented to it; a script arrives inside a workspace, as data, with no such
|
||||
* moment, which is why scripts will never get a runtime other than this one.
|
||||
*/
|
||||
|
||||
import type { PluginDefinition } from "@yaakapp/api";
|
||||
import {
|
||||
applyFormInputDefaults,
|
||||
validateTemplateFunctionArgs,
|
||||
} from "@yaakapp-internal/lib/templateFunction";
|
||||
import {
|
||||
applyDynamicFormInput,
|
||||
migrateTemplateFunctionSelectOptions,
|
||||
stripDynamicCallbacks,
|
||||
} from "@yaakapp-internal/lib/pluginForms";
|
||||
import type {
|
||||
GrpcRequestAction,
|
||||
HttpAuthenticationAction,
|
||||
HttpRequestAction,
|
||||
ImportResources,
|
||||
InternalEventPayload,
|
||||
PluginContext,
|
||||
TemplateFunction,
|
||||
} from "@yaakapp-internal/plugins";
|
||||
import { newContext } from "./context";
|
||||
import { installGlobals } from "./globals";
|
||||
|
||||
declare const __yaak_call: (payloadJson: string) => Promise<string>;
|
||||
|
||||
const { fireTimer } = installGlobals();
|
||||
|
||||
/** The loaded module, and the id the host knows it by. */
|
||||
let mod: PluginDefinition = {};
|
||||
let pluginRefId = "";
|
||||
|
||||
/**
|
||||
* Evaluate a module's source.
|
||||
*
|
||||
* The bundles are CommonJS, so they are handed the three names that implies and
|
||||
* nothing else. `require` is the interesting one: it exists only to fail, by
|
||||
* name, because a bundle that still calls it did not get bundled for this
|
||||
* target and the honest outcome is a message saying which specifier is missing
|
||||
* rather than an undefined that surfaces ten frames later.
|
||||
*/
|
||||
function load(source: string, refId: string): void {
|
||||
const module: { exports: Record<string, unknown> } = { exports: {} };
|
||||
const require = (specifier: string) => {
|
||||
throw new Error(
|
||||
`Module "${specifier}" is not available in the sandbox runtime. ` +
|
||||
`Plugins must be bundled with no external or built-in modules.`,
|
||||
);
|
||||
};
|
||||
|
||||
// `new Function` rather than an ES module so the bundle's own top-level names
|
||||
// cannot collide with this shell's, and so the source can arrive as a string
|
||||
// with no loader hook. Evaluating untrusted source is the entire job of this
|
||||
// file; the isolation is the QuickJS context around it, not a lint rule.
|
||||
// oxlint-disable-next-line no-implied-eval
|
||||
const factory = new Function("module", "exports", "require", source);
|
||||
factory(module, module.exports, require);
|
||||
|
||||
const loaded = (module.exports.plugin ?? module.exports.default) as PluginDefinition | undefined;
|
||||
if (loaded == null || typeof loaded !== "object") {
|
||||
throw new Error("Module did not export `plugin`");
|
||||
}
|
||||
mod = loaded;
|
||||
pluginRefId = refId;
|
||||
}
|
||||
|
||||
/** Everything a module contributes, without the functions that implement it. */
|
||||
function summary(): Record<string, unknown> {
|
||||
return {
|
||||
templateFunctions: (mod.templateFunctions ?? []).map((f) => f.name),
|
||||
authentication: mod.authentication?.name ?? null,
|
||||
importer: mod.importer != null,
|
||||
filter: mod.filter != null,
|
||||
themes: (mod.themes ?? []).length,
|
||||
httpRequestActions: (mod.httpRequestActions ?? []).length,
|
||||
workspaceActions: (mod.workspaceActions ?? []).length,
|
||||
folderActions: (mod.folderActions ?? []).length,
|
||||
grpcRequestActions: (mod.grpcRequestActions ?? []).length,
|
||||
websocketRequestActions: (mod.websocketRequestActions ?? []).length,
|
||||
};
|
||||
}
|
||||
|
||||
const EMPTY: InternalEventPayload = { type: "empty_response" };
|
||||
|
||||
/**
|
||||
* Answer one event against the loaded module.
|
||||
*
|
||||
* Every branch mirrors the Node runtime's, because the payloads are the same
|
||||
* payloads — a plugin cannot tell which runtime it is in, and that is the
|
||||
* promise the whole design exists to keep. An unmatched event gets
|
||||
* `empty_response` rather than silence, so a caller never waits forever for a
|
||||
* capability this module doesn't have.
|
||||
*/
|
||||
async function dispatch(
|
||||
context: PluginContext,
|
||||
payload: InternalEventPayload,
|
||||
): Promise<InternalEventPayload> {
|
||||
const ctx = newContext(hostCall, context);
|
||||
|
||||
if (payload.type === "boot_request") {
|
||||
await mod.init?.(ctx);
|
||||
return { type: "boot_response" };
|
||||
}
|
||||
|
||||
if (payload.type === "terminate_request") {
|
||||
await mod.dispose?.();
|
||||
return { type: "terminate_response" };
|
||||
}
|
||||
|
||||
if (payload.type === "import_request" && typeof mod.importer?.onImport === "function") {
|
||||
const reply = await mod.importer.onImport(ctx, { text: payload.content });
|
||||
if (reply != null) {
|
||||
return { type: "import_response", resources: reply.resources as ImportResources };
|
||||
}
|
||||
return EMPTY;
|
||||
}
|
||||
|
||||
if (payload.type === "filter_request" && typeof mod.filter?.onFilter === "function") {
|
||||
const reply = await mod.filter.onFilter(ctx, {
|
||||
filter: payload.filter,
|
||||
payload: payload.content,
|
||||
mimeType: payload.type,
|
||||
});
|
||||
return { type: "filter_response", ...reply };
|
||||
}
|
||||
|
||||
if (payload.type === "get_themes_request" && Array.isArray(mod.themes)) {
|
||||
return { type: "get_themes_response", themes: mod.themes };
|
||||
}
|
||||
|
||||
/* --------------------------- template functions -------------------------- */
|
||||
|
||||
if (
|
||||
payload.type === "get_template_function_summary_request" &&
|
||||
Array.isArray(mod.templateFunctions)
|
||||
) {
|
||||
const functions: TemplateFunction[] = mod.templateFunctions.map((f) => ({
|
||||
...migrateTemplateFunctionSelectOptions(f),
|
||||
onRender: undefined,
|
||||
}));
|
||||
return { type: "get_template_function_summary_response", pluginRefId, functions };
|
||||
}
|
||||
|
||||
if (
|
||||
payload.type === "get_template_function_config_request" &&
|
||||
Array.isArray(mod.templateFunctions)
|
||||
) {
|
||||
const found = mod.templateFunctions.find((f) => f.name === payload.name);
|
||||
if (found == null) return EMPTY;
|
||||
|
||||
const fn = { ...migrateTemplateFunctionSelectOptions(found), onRender: undefined };
|
||||
payload.values = applyFormInputDefaults(fn.args, payload.values);
|
||||
const resolved = await applyDynamicFormInput(ctx, fn.args, {
|
||||
...payload,
|
||||
purpose: "preview",
|
||||
} as const);
|
||||
|
||||
return {
|
||||
type: "get_template_function_config_response",
|
||||
pluginRefId,
|
||||
function: { ...fn, args: stripDynamicCallbacks(resolved) },
|
||||
};
|
||||
}
|
||||
|
||||
if (payload.type === "call_template_function_request" && Array.isArray(mod.templateFunctions)) {
|
||||
const fn = mod.templateFunctions.find((f) => f.name === payload.name);
|
||||
|
||||
if (
|
||||
payload.args.purpose === "preview" &&
|
||||
(fn?.previewType === "click" || fn?.previewType === "none")
|
||||
) {
|
||||
return {
|
||||
type: "call_template_function_response",
|
||||
value: null,
|
||||
error: "Live preview disabled for this function",
|
||||
};
|
||||
}
|
||||
|
||||
if (typeof fn?.onRender === "function") {
|
||||
const resolved = await applyDynamicFormInput(ctx, fn.args, payload.args);
|
||||
const values = applyFormInputDefaults(resolved, payload.args.values);
|
||||
const error = validateTemplateFunctionArgs(fn.name, resolved, values);
|
||||
if (error && payload.args.purpose !== "preview") {
|
||||
return { type: "call_template_function_response", value: null, error };
|
||||
}
|
||||
|
||||
const result = await fn.onRender(ctx, { ...payload.args, values });
|
||||
return { type: "call_template_function_response", value: result ?? null };
|
||||
}
|
||||
}
|
||||
|
||||
/* --------------------------- http authentication ------------------------- */
|
||||
|
||||
if (payload.type === "get_http_authentication_summary_request" && mod.authentication) {
|
||||
return { type: "get_http_authentication_summary_response", ...mod.authentication };
|
||||
}
|
||||
|
||||
if (payload.type === "get_http_authentication_config_request" && mod.authentication) {
|
||||
const { args, actions } = mod.authentication;
|
||||
payload.values = applyFormInputDefaults(args, payload.values);
|
||||
const resolved = await applyDynamicFormInput(ctx, args, payload);
|
||||
const resolvedActions: HttpAuthenticationAction[] = [];
|
||||
// oxlint-disable-next-line unbound-method
|
||||
for (const { onSelect: _onSelect, ...action } of actions ?? []) resolvedActions.push(action);
|
||||
|
||||
return {
|
||||
type: "get_http_authentication_config_response",
|
||||
args: stripDynamicCallbacks(resolved),
|
||||
actions: resolvedActions,
|
||||
pluginRefId,
|
||||
};
|
||||
}
|
||||
|
||||
if (payload.type === "call_http_authentication_request" && mod.authentication) {
|
||||
const auth = mod.authentication;
|
||||
if (typeof auth.onApply === "function") {
|
||||
const resolved = await applyDynamicFormInput(ctx, auth.args, payload);
|
||||
payload.values = applyFormInputDefaults(resolved, payload.values);
|
||||
return { type: "call_http_authentication_response", ...(await auth.onApply(ctx, payload)) };
|
||||
}
|
||||
}
|
||||
|
||||
if (payload.type === "call_http_authentication_action_request" && mod.authentication != null) {
|
||||
const action = mod.authentication.actions?.[payload.index];
|
||||
if (typeof action?.onSelect === "function") {
|
||||
await action.onSelect(ctx, payload.args);
|
||||
return EMPTY;
|
||||
}
|
||||
}
|
||||
|
||||
/* --------------------------------- actions ------------------------------- */
|
||||
|
||||
if (payload.type === "get_http_request_actions_request" && Array.isArray(mod.httpRequestActions)) {
|
||||
const actions: HttpRequestAction[] = mod.httpRequestActions.map((a) => ({
|
||||
...a,
|
||||
onSelect: undefined,
|
||||
}));
|
||||
return { type: "get_http_request_actions_response", pluginRefId, actions };
|
||||
}
|
||||
|
||||
if (
|
||||
payload.type === "get_websocket_request_actions_request" &&
|
||||
Array.isArray(mod.websocketRequestActions)
|
||||
) {
|
||||
const actions = mod.websocketRequestActions.map((a) => ({ ...a, onSelect: undefined }));
|
||||
return { type: "get_websocket_request_actions_response", pluginRefId, actions };
|
||||
}
|
||||
|
||||
if (payload.type === "get_grpc_request_actions_request" && Array.isArray(mod.grpcRequestActions)) {
|
||||
const actions: GrpcRequestAction[] = mod.grpcRequestActions.map((a) => ({
|
||||
...a,
|
||||
onSelect: undefined,
|
||||
}));
|
||||
return { type: "get_grpc_request_actions_response", pluginRefId, actions };
|
||||
}
|
||||
|
||||
if (payload.type === "get_workspace_actions_request" && Array.isArray(mod.workspaceActions)) {
|
||||
const actions = mod.workspaceActions.map((a) => ({ ...a, onSelect: undefined }));
|
||||
return { type: "get_workspace_actions_response", pluginRefId, actions };
|
||||
}
|
||||
|
||||
if (payload.type === "get_folder_actions_request" && Array.isArray(mod.folderActions)) {
|
||||
const actions = mod.folderActions.map((a) => ({ ...a, onSelect: undefined }));
|
||||
return { type: "get_folder_actions_response", pluginRefId, actions };
|
||||
}
|
||||
|
||||
const called = await callAction(ctx, payload);
|
||||
if (called) return EMPTY;
|
||||
|
||||
return EMPTY;
|
||||
}
|
||||
|
||||
/** The five action kinds, which differ only in which list they index into. */
|
||||
async function callAction(
|
||||
ctx: ReturnType<typeof newContext>,
|
||||
payload: InternalEventPayload,
|
||||
): Promise<boolean> {
|
||||
const lists = {
|
||||
call_http_request_action_request: mod.httpRequestActions,
|
||||
call_websocket_request_action_request: mod.websocketRequestActions,
|
||||
call_grpc_request_action_request: mod.grpcRequestActions,
|
||||
call_workspace_action_request: mod.workspaceActions,
|
||||
call_folder_action_request: mod.folderActions,
|
||||
} as const;
|
||||
|
||||
const list = lists[payload.type as keyof typeof lists];
|
||||
if (!Array.isArray(list)) return false;
|
||||
|
||||
const action = list[(payload as { index: number }).index];
|
||||
if (typeof action?.onSelect !== "function") return false;
|
||||
|
||||
await action.onSelect(ctx, (payload as { args: never }).args);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** One outgoing request, JSON out and JSON back. */
|
||||
async function hostCall(
|
||||
context: PluginContext,
|
||||
payload: InternalEventPayload,
|
||||
): Promise<Record<string, unknown>> {
|
||||
// The id rides along because the host multiplexes every loaded module
|
||||
// through one handler, and a plugin's storage is namespaced by which plugin
|
||||
// it is.
|
||||
const replyJson = await __yaak_call(JSON.stringify({ pluginRefId, context, payload }));
|
||||
const reply = JSON.parse(replyJson) as InternalEventPayload & { error?: string };
|
||||
if (reply.type === "error_response") {
|
||||
throw new Error(reply.error || `Host failed to handle ${payload.type}`);
|
||||
}
|
||||
const { type: _type, ...rest } = reply;
|
||||
return rest as Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* What the host can reach.
|
||||
*
|
||||
* Named on `globalThis` because the host calls them by evaluating an
|
||||
* expression, and kept to four: load a module, ask what it has, send it an
|
||||
* event, wake a timer.
|
||||
*/
|
||||
(globalThis as Record<string, unknown>).__yaak_guest = {
|
||||
load,
|
||||
summary,
|
||||
fireTimer,
|
||||
dispatch: async (envelopeJson: string): Promise<string> => {
|
||||
const { context, payload } = JSON.parse(envelopeJson) as {
|
||||
context: PluginContext;
|
||||
payload: InternalEventPayload;
|
||||
};
|
||||
try {
|
||||
return JSON.stringify(await dispatch(context, payload));
|
||||
} catch (err) {
|
||||
// A throw from inside a plugin is an answer, not a crash: the host turns
|
||||
// it into the same `error_response` the Node runtime sends, and whatever
|
||||
// asked for this gets a message instead of a hang.
|
||||
const error = (err instanceof Error ? err.message : String(err)).replace(/^Error:\s*/g, "");
|
||||
return JSON.stringify({ type: "error_response", error });
|
||||
}
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,384 @@
|
||||
/**
|
||||
* The sandbox host: QuickJS, and the four things that cross into it.
|
||||
*
|
||||
* One QuickJS runtime holds one context per loaded module. A context is the
|
||||
* isolation boundary — its own globals, its own `Object`, its own prototypes —
|
||||
* so two plugins cannot see or patch each other, and neither can reach the
|
||||
* worker's own scope. Sharing a runtime between them is deliberate: the engine
|
||||
* and its wasm instance are the expensive part, contexts are not.
|
||||
*
|
||||
* The engine is `quickjs-ng`, not Bellard's, and the sync variant rather than
|
||||
* the ASYNCIFY one. Both choices are recorded in this package's README along
|
||||
* with what they cost; the short version is that the Rust host has no choice
|
||||
* (rquickjs vendors quickjs-ng and offers no alternative), and the sync build
|
||||
* still gives the guest real `await` through a deferred promise, at half the
|
||||
* size and twice the speed.
|
||||
*/
|
||||
|
||||
import variant from "@jitl/quickjs-ng-wasmfile-release-sync";
|
||||
import {
|
||||
newQuickJSWASMModuleFromVariant,
|
||||
type QuickJSContext,
|
||||
type QuickJSRuntime,
|
||||
type QuickJSWASMModule,
|
||||
} from "quickjs-emscripten-core";
|
||||
import { GUEST_SOURCE } from "../generated/guest";
|
||||
|
||||
/**
|
||||
* What a module may allocate.
|
||||
*
|
||||
* Sized for the job rather than for comfort: an importer holding a large spec
|
||||
* and the objects it parses into is the high-water mark, and a plugin that
|
||||
* wants more than this is doing something a plugin should not. Hitting it
|
||||
* throws inside the sandbox and unwinds as an ordinary error.
|
||||
*/
|
||||
const MEMORY_LIMIT_BYTES = 256 * 1024 * 1024;
|
||||
|
||||
/** Deep recursion is a stack overflow inside the guest, not a crash of the worker. */
|
||||
const STACK_SIZE_BYTES = 2 * 1024 * 1024;
|
||||
|
||||
/**
|
||||
* How long a module may run without yielding.
|
||||
*
|
||||
* This bounds *synchronous* execution only, and it has to: a plugin awaiting
|
||||
* the host is not looping, it is waiting for us. So the clock is set when a
|
||||
* dispatch begins and pushed back whenever the guest hands control back, which
|
||||
* makes it a watchdog for `while (true)` rather than a limit on how long real
|
||||
* work may take.
|
||||
*
|
||||
* Generous, because it costs nothing to be: a plugin runs in its own worker,
|
||||
* so one stuck here blocks no database command and no frame. It is sized off
|
||||
* the slowest real work measured (`bench/import.mjs`: GitHub's 12 MB OpenAPI
|
||||
* description takes about four seconds), with room for a document several
|
||||
* times larger before a legitimate import looks like a hang.
|
||||
*/
|
||||
const SYNC_BUDGET_MS = 60_000;
|
||||
|
||||
export type HostRequestHandler = (envelopeJson: string) => Promise<string>;
|
||||
|
||||
export interface SandboxLog {
|
||||
pluginRefId: string;
|
||||
level: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
let modulePromise: Promise<QuickJSWASMModule> | null = null;
|
||||
|
||||
function quickjs(): Promise<QuickJSWASMModule> {
|
||||
// Loaded once per worker, on first use. The wasm is ~529 KB and there is no
|
||||
// reason to pay for it in a session where nothing calls a plugin.
|
||||
modulePromise ??= newQuickJSWASMModuleFromVariant(variant);
|
||||
return modulePromise;
|
||||
}
|
||||
|
||||
/** One loaded module, and the context it lives in. */
|
||||
class LoadedPlugin {
|
||||
readonly pluginRefId: string;
|
||||
readonly context: QuickJSContext;
|
||||
/** Set while a dispatch is running; the interrupt handler reads it. */
|
||||
deadline: number | null = null;
|
||||
private nextTimer = new Map<number, ReturnType<typeof setTimeout>>();
|
||||
private disposed = false;
|
||||
|
||||
constructor(pluginRefId: string, context: QuickJSContext) {
|
||||
this.pluginRefId = pluginRefId;
|
||||
this.context = context;
|
||||
}
|
||||
|
||||
/** Push the synchronous-execution deadline back; called whenever the guest yields. */
|
||||
touch(): void {
|
||||
if (this.deadline != null) this.deadline = Date.now() + SYNC_BUDGET_MS;
|
||||
}
|
||||
|
||||
startTimer(id: number, ms: number, fire: () => void): void {
|
||||
this.nextTimer.set(
|
||||
id,
|
||||
setTimeout(() => {
|
||||
this.nextTimer.delete(id);
|
||||
if (!this.disposed) fire();
|
||||
}, ms),
|
||||
);
|
||||
}
|
||||
|
||||
cancelTimer(id: number): void {
|
||||
const handle = this.nextTimer.get(id);
|
||||
if (handle == null) return;
|
||||
clearTimeout(handle);
|
||||
this.nextTimer.delete(id);
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
if (this.disposed) return;
|
||||
this.disposed = true;
|
||||
for (const handle of this.nextTimer.values()) clearTimeout(handle);
|
||||
this.nextTimer.clear();
|
||||
this.context.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
export class PluginSandboxHost {
|
||||
private runtime: QuickJSRuntime | null = null;
|
||||
private readonly plugins = new Map<string, LoadedPlugin>();
|
||||
|
||||
constructor(
|
||||
private readonly onHostRequest: HostRequestHandler,
|
||||
private readonly onLog: (log: SandboxLog) => void,
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Load one module's source under an id.
|
||||
*
|
||||
* Replaces whatever was loaded under that id, disposing it first, so a
|
||||
* reload is a fresh context rather than a re-evaluation on top of the old
|
||||
* one's globals.
|
||||
*/
|
||||
async load(pluginRefId: string, source: string): Promise<Record<string, unknown>> {
|
||||
const module = await quickjs();
|
||||
|
||||
if (this.runtime == null) {
|
||||
this.runtime = module.newRuntime();
|
||||
this.runtime.setMemoryLimit(MEMORY_LIMIT_BYTES);
|
||||
this.runtime.setMaxStackSize(STACK_SIZE_BYTES);
|
||||
// One handler for every context on the runtime. A plugin that is merely
|
||||
// waiting has no deadline set, so it is never interrupted.
|
||||
this.runtime.setInterruptHandler(() => {
|
||||
const now = Date.now();
|
||||
for (const plugin of this.plugins.values()) {
|
||||
if (plugin.deadline != null && now > plugin.deadline) return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
}
|
||||
|
||||
this.plugins.get(pluginRefId)?.dispose();
|
||||
|
||||
const plugin = new LoadedPlugin(pluginRefId, this.runtime.newContext());
|
||||
this.plugins.set(pluginRefId, plugin);
|
||||
|
||||
try {
|
||||
this.installHostFunctions(plugin);
|
||||
this.evalOrThrow(plugin, GUEST_SOURCE, "yaak:sandbox-shell");
|
||||
await this.callGuest(plugin, "load", [source, pluginRefId]);
|
||||
return await this.callGuest(plugin, "summary", []);
|
||||
} catch (err) {
|
||||
plugin.dispose();
|
||||
this.plugins.delete(pluginRefId);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
loaded(): string[] {
|
||||
return Array.from(this.plugins.keys());
|
||||
}
|
||||
|
||||
unload(pluginRefId: string): void {
|
||||
this.plugins.get(pluginRefId)?.dispose();
|
||||
this.plugins.delete(pluginRefId);
|
||||
}
|
||||
|
||||
/** Send one event to one loaded module and wait for its reply payload. */
|
||||
async dispatch(pluginRefId: string, envelopeJson: string): Promise<string> {
|
||||
const plugin = this.plugins.get(pluginRefId);
|
||||
if (plugin == null) throw new Error(`No plugin loaded as \`${pluginRefId}\``);
|
||||
const reply = await this.callGuest(plugin, "dispatch", [envelopeJson]);
|
||||
return reply as unknown as string;
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
for (const plugin of this.plugins.values()) plugin.dispose();
|
||||
this.plugins.clear();
|
||||
this.runtime?.dispose();
|
||||
this.runtime = null;
|
||||
}
|
||||
|
||||
/* ------------------------------ internals ------------------------------- */
|
||||
|
||||
private installHostFunctions(plugin: LoadedPlugin): void {
|
||||
const { context } = plugin;
|
||||
|
||||
const define = (name: string, fn: Parameters<QuickJSContext["newFunction"]>[1]) => {
|
||||
const handle = context.newFunction(name, fn);
|
||||
context.setProp(context.global, name, handle);
|
||||
handle.dispose();
|
||||
};
|
||||
|
||||
define("__yaak_log", (levelHandle, messageHandle) => {
|
||||
this.onLog({
|
||||
pluginRefId: plugin.pluginRefId,
|
||||
level: context.getString(levelHandle),
|
||||
message: context.getString(messageHandle),
|
||||
});
|
||||
});
|
||||
|
||||
define("__yaak_timer_start", (idHandle, msHandle) => {
|
||||
const id = context.getNumber(idHandle);
|
||||
plugin.startTimer(id, context.getNumber(msHandle), () => {
|
||||
// Waking a timer re-enters the guest, so it gets a fresh budget.
|
||||
plugin.touch();
|
||||
this.callGuestSync(plugin, "fireTimer", [id]);
|
||||
this.pump(plugin);
|
||||
});
|
||||
});
|
||||
|
||||
define("__yaak_timer_cancel", (idHandle) => {
|
||||
plugin.cancelTimer(context.getNumber(idHandle));
|
||||
});
|
||||
|
||||
// The one door out. Everything a plugin does to the world arrives here as
|
||||
// a JSON envelope and leaves as a JSON reply; the guest's whole `ctx` is
|
||||
// built from this single function.
|
||||
define("__yaak_call", (envelopeHandle) => {
|
||||
const envelope = context.getString(envelopeHandle);
|
||||
|
||||
// While the host answers, the guest is suspended, not looping — so the
|
||||
// watchdog stops until it comes back.
|
||||
const wasWatching = plugin.deadline != null;
|
||||
plugin.deadline = null;
|
||||
|
||||
const settle = this.onHostRequest(envelope).then(
|
||||
(reply) => {
|
||||
if (wasWatching) plugin.touch();
|
||||
return context.newString(reply);
|
||||
},
|
||||
(err: unknown) => {
|
||||
if (wasWatching) plugin.touch();
|
||||
// Rejections come back as an error the guest can catch, which is
|
||||
// what a host that cannot answer should look like from inside.
|
||||
return context.newError(err instanceof Error ? err.message : String(err));
|
||||
},
|
||||
);
|
||||
|
||||
const deferred = context.newPromise(settle);
|
||||
// Resolving a promise only queues its reactions; something has to run
|
||||
// them, and inside a sandbox that something is us.
|
||||
void deferred.settled.then(() => {
|
||||
this.pump(plugin);
|
||||
deferred.dispose();
|
||||
});
|
||||
return deferred.handle;
|
||||
});
|
||||
}
|
||||
|
||||
/** Drain the guest's microtask queue. */
|
||||
private pump(plugin: LoadedPlugin): void {
|
||||
const result = this.runtime?.executePendingJobs();
|
||||
if (result?.error != null) {
|
||||
this.onLog({
|
||||
pluginRefId: plugin.pluginRefId,
|
||||
level: "error",
|
||||
message: `Unhandled error in sandbox: ${result.error.consume(
|
||||
plugin.context.dump.bind(plugin.context),
|
||||
)}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private evalOrThrow(plugin: LoadedPlugin, source: string, filename: string): void {
|
||||
plugin.deadline = Date.now() + SYNC_BUDGET_MS;
|
||||
try {
|
||||
const result = plugin.context.evalCode(source, filename);
|
||||
if (result.error != null) {
|
||||
throw this.toError(plugin, result.error.consume(plugin.context.dump.bind(plugin.context)));
|
||||
}
|
||||
result.value.dispose();
|
||||
} finally {
|
||||
plugin.deadline = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Call `__yaak_guest.<method>(...args)`, awaiting the result if it is a promise. */
|
||||
private async callGuest(
|
||||
plugin: LoadedPlugin,
|
||||
method: string,
|
||||
args: (string | number)[],
|
||||
// oxlint-disable-next-line no-explicit-any -- the caller knows the guest's shape
|
||||
): Promise<any> {
|
||||
const { context } = plugin;
|
||||
plugin.deadline = Date.now() + SYNC_BUDGET_MS;
|
||||
|
||||
const guest = context.getProp(context.global, "__yaak_guest");
|
||||
const fn = context.getProp(guest, method);
|
||||
const argHandles = args.map((a) =>
|
||||
typeof a === "string" ? context.newString(a) : context.newNumber(a),
|
||||
);
|
||||
|
||||
try {
|
||||
const called = context.callFunction(fn, guest, ...argHandles);
|
||||
if (called.error != null) {
|
||||
throw this.toError(plugin, called.error.consume(context.dump.bind(context)));
|
||||
}
|
||||
|
||||
const value = called.value;
|
||||
const state = context.getPromiseState(value);
|
||||
if (state.type !== "fulfilled" || state.notAPromise !== true) {
|
||||
// A promise: hand control back so the guest can make progress, then
|
||||
// wait for it on this side.
|
||||
const resolved = context.resolvePromise(value);
|
||||
value.dispose();
|
||||
this.pump(plugin);
|
||||
const settled = await resolved;
|
||||
if (settled.error != null) {
|
||||
throw this.toError(plugin, settled.error.consume(context.dump.bind(context)));
|
||||
}
|
||||
return settled.value.consume(context.dump.bind(context));
|
||||
}
|
||||
|
||||
return value.consume(context.dump.bind(context));
|
||||
} finally {
|
||||
plugin.deadline = null;
|
||||
for (const handle of argHandles) handle.dispose();
|
||||
fn.dispose();
|
||||
guest.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
/** The timer path: fire and forget, because nothing is waiting on it. */
|
||||
private callGuestSync(plugin: LoadedPlugin, method: string, args: number[]): void {
|
||||
const { context } = plugin;
|
||||
const guest = context.getProp(context.global, "__yaak_guest");
|
||||
const fn = context.getProp(guest, method);
|
||||
const argHandles = args.map((a) => context.newNumber(a));
|
||||
try {
|
||||
const called = context.callFunction(fn, guest, ...argHandles);
|
||||
if (called.error != null) {
|
||||
this.onLog({
|
||||
pluginRefId: plugin.pluginRefId,
|
||||
level: "error",
|
||||
message: String(this.toError(plugin, called.error.consume(context.dump.bind(context)))),
|
||||
});
|
||||
} else {
|
||||
called.value.dispose();
|
||||
}
|
||||
} finally {
|
||||
for (const handle of argHandles) handle.dispose();
|
||||
fn.dispose();
|
||||
guest.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A dumped QuickJS error as a host `Error`.
|
||||
*
|
||||
* The guest's stack is kept in the message: it names lines in the plugin's
|
||||
* own bundle, which is the only stack that means anything to whoever wrote
|
||||
* it — the worker's own stack would just say "sandbox.ts".
|
||||
*/
|
||||
private toError(plugin: LoadedPlugin, dumped: unknown): Error {
|
||||
if (dumped != null && typeof dumped === "object") {
|
||||
const { message, name, stack } = dumped as Record<string, string | undefined>;
|
||||
const error = new Error(message ?? JSON.stringify(dumped));
|
||||
if (name != null) error.name = name;
|
||||
if (stack != null) error.stack = `${name ?? "Error"}: ${message ?? ""}\n${stack}`;
|
||||
return error;
|
||||
}
|
||||
// An interrupted plugin surfaces as `null` with no error object at all,
|
||||
// which would otherwise read as a mysterious empty failure.
|
||||
if (dumped == null) {
|
||||
return new Error(
|
||||
`Plugin \`${plugin.pluginRefId}\` was stopped after running for ` +
|
||||
`${SYNC_BUDGET_MS / 1000}s without yielding`,
|
||||
);
|
||||
}
|
||||
return new Error(typeof dumped === "string" ? dumped : JSON.stringify(dumped));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* A tab's handle on its sandbox.
|
||||
*
|
||||
* Owns the worker, keeps track of what is loaded in it, and turns the two
|
||||
* message flows into promises. The interesting half is `onHostRequest`: the
|
||||
* caller supplies it, and it is the entire answer to "what can a plugin do
|
||||
* here?" — this package deliberately has no idea. A browser host answers those
|
||||
* against a wasm database and a send proxy; something else could answer them
|
||||
* differently; a host that answers nothing still runs plugins that only compute.
|
||||
*/
|
||||
|
||||
import type { FromSandbox, ToSandbox } from "./protocol";
|
||||
|
||||
|
||||
/** Answers one `ctx` call. Gets the JSON envelope, returns the JSON reply. */
|
||||
export type HostRequestHandler = (envelope: string) => Promise<string>;
|
||||
|
||||
export interface PluginSandboxOptions {
|
||||
onHostRequest: HostRequestHandler;
|
||||
onLog?: (log: { pluginRefId: string; level: string; message: string }) => void;
|
||||
}
|
||||
|
||||
/** What a module turned out to contribute, as reported after loading. */
|
||||
export interface PluginSummary {
|
||||
templateFunctions: string[];
|
||||
authentication: string | null;
|
||||
importer: boolean;
|
||||
filter: boolean;
|
||||
themes: number;
|
||||
httpRequestActions: number;
|
||||
workspaceActions: number;
|
||||
folderActions: number;
|
||||
grpcRequestActions: number;
|
||||
websocketRequestActions: number;
|
||||
}
|
||||
|
||||
type Pending = { resolve: (value: unknown) => void; reject: (reason: Error) => void };
|
||||
|
||||
export class PluginSandbox {
|
||||
private readonly worker: Worker;
|
||||
private readonly pending = new Map<number, Pending>();
|
||||
private readonly options: PluginSandboxOptions;
|
||||
private nextId = 1;
|
||||
|
||||
constructor(options: PluginSandboxOptions) {
|
||||
this.options = options;
|
||||
|
||||
// `new URL("./worker.ts", import.meta.url)` is written inline because that
|
||||
// exact syntax is what the bundler pattern-matches to know it must bundle a
|
||||
// worker entry. Hoisted into a variable it ships as raw TypeScript.
|
||||
this.worker = new Worker(new URL("./worker.ts", import.meta.url), {
|
||||
type: "module",
|
||||
name: "yaak-plugins",
|
||||
});
|
||||
this.worker.onmessage = (e: MessageEvent<FromSandbox>) => this.receive(e.data);
|
||||
this.worker.onerror = () => this.failEverything("The plugin sandbox failed to start");
|
||||
}
|
||||
|
||||
/** Load a module's source under an id, replacing anything already there. */
|
||||
load(pluginRefId: string, source: string): Promise<PluginSummary> {
|
||||
return this.request<PluginSummary>((id) => ({ type: "load", id, pluginRefId, source }));
|
||||
}
|
||||
|
||||
unload(pluginRefId: string): Promise<void> {
|
||||
return this.request<void>((id) => ({ type: "unload", id, pluginRefId }));
|
||||
}
|
||||
|
||||
/** Send one event to one loaded module; resolves with its reply payload. */
|
||||
async dispatch<T>(
|
||||
pluginRefId: string,
|
||||
context: unknown,
|
||||
payload: unknown,
|
||||
): Promise<T & { type: string }> {
|
||||
const envelope = JSON.stringify({ context, payload });
|
||||
const reply = await this.request<string>((id) => ({
|
||||
type: "dispatch",
|
||||
id,
|
||||
pluginRefId,
|
||||
envelope,
|
||||
}));
|
||||
const parsed = JSON.parse(reply) as { type: string; error?: string };
|
||||
if (parsed.type === "error_response") {
|
||||
throw new Error(parsed.error || "Plugin failed");
|
||||
}
|
||||
return parsed as T & { type: string };
|
||||
}
|
||||
|
||||
/**
|
||||
* End the sandbox now.
|
||||
*
|
||||
* `terminate()` rather than a polite shutdown, on purpose: the reason to
|
||||
* reach for this is a plugin that will not stop, and asking it to stop is
|
||||
* exactly what does not work then.
|
||||
*/
|
||||
dispose(): void {
|
||||
this.worker.terminate();
|
||||
this.failEverything("The plugin sandbox was shut down");
|
||||
}
|
||||
|
||||
/* ------------------------------ internals ------------------------------- */
|
||||
|
||||
private request<T>(build: (id: number) => ToSandbox): Promise<T> {
|
||||
const id = this.nextId++;
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
this.pending.set(id, { resolve: resolve as (v: unknown) => void, reject });
|
||||
this.worker.postMessage(build(id));
|
||||
});
|
||||
}
|
||||
|
||||
private receive(message: FromSandbox): void {
|
||||
switch (message.type) {
|
||||
case "result": {
|
||||
const p = this.pending.get(message.id);
|
||||
this.pending.delete(message.id);
|
||||
p?.resolve(message.result);
|
||||
return;
|
||||
}
|
||||
case "error": {
|
||||
const p = this.pending.get(message.id);
|
||||
this.pending.delete(message.id);
|
||||
p?.reject(new Error(message.message));
|
||||
return;
|
||||
}
|
||||
case "log":
|
||||
this.options.onLog?.(message);
|
||||
return;
|
||||
case "host_call":
|
||||
void this.answer(message.id, message.envelope);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
private async answer(id: number, envelope: string): Promise<void> {
|
||||
let reply: ToSandbox;
|
||||
try {
|
||||
reply = { type: "host_result", id, reply: await this.options.onHostRequest(envelope) };
|
||||
} catch (err) {
|
||||
reply = {
|
||||
type: "host_result",
|
||||
id,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
};
|
||||
}
|
||||
this.worker.postMessage(reply);
|
||||
}
|
||||
|
||||
private failEverything(message: string): void {
|
||||
for (const [id, p] of this.pending) {
|
||||
this.pending.delete(id);
|
||||
p.reject(new Error(message));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* The messages between a tab and its sandbox worker.
|
||||
*
|
||||
* Two request/reply flows in opposite directions. The tab asks the worker to
|
||||
* load a module or send it an event; the worker asks the tab to answer a
|
||||
* plugin's `ctx` call, because the tab is the only side with a database, a
|
||||
* network and a user. Both carry payloads as JSON strings rather than objects:
|
||||
* they have to be strings to cross into QuickJS anyway, and serializing once at
|
||||
* the edge is cheaper than structured-cloning an object the worker will only
|
||||
* stringify again.
|
||||
*/
|
||||
|
||||
/** Tab → worker */
|
||||
export type ToSandbox =
|
||||
| { type: "load"; id: number; pluginRefId: string; source: string }
|
||||
| { type: "unload"; id: number; pluginRefId: string }
|
||||
| { type: "dispatch"; id: number; pluginRefId: string; envelope: string }
|
||||
/** The tab's answer to a `host_call`. */
|
||||
| { type: "host_result"; id: number; reply?: string; error?: string };
|
||||
|
||||
/** Worker → tab */
|
||||
export type FromSandbox =
|
||||
| { type: "result"; id: number; result: unknown }
|
||||
| { type: "error"; id: number; message: string }
|
||||
/** A plugin wants something only the tab can provide. */
|
||||
| { type: "host_call"; id: number; envelope: string }
|
||||
| { type: "log"; pluginRefId: string; level: string; message: string };
|
||||
@@ -0,0 +1,82 @@
|
||||
/// <reference lib="webworker" />
|
||||
|
||||
/**
|
||||
* The worker plugins run in.
|
||||
*
|
||||
* A dedicated worker, owned by the tab that made it — deliberately not the
|
||||
* SharedWorker that owns the database, for three reasons. Plugin work is slow
|
||||
* by design (see `bench/import.mjs`) and the database worker answers every
|
||||
* tab's commands synchronously, so a large import in there would stall every
|
||||
* other tab's reads. A plugin that never returns can be ended with
|
||||
* `terminate()`, which is not something you can do to the worker holding the
|
||||
* database. And the capabilities a plugin actually asks for — a prompt, a
|
||||
* toast, the active request — belong to a tab rather than to a database, so
|
||||
* routing through the tab is the shorter path anyway, not a detour.
|
||||
*
|
||||
* That leaves the database one hop further away than it would otherwise be:
|
||||
* `ctx.store` goes worker → tab → database worker. It is a message either way,
|
||||
* and this direction is the one where a stuck plugin costs nothing.
|
||||
*/
|
||||
|
||||
import { PluginSandboxHost } from "./host/sandbox";
|
||||
import type { FromSandbox, ToSandbox } from "./protocol";
|
||||
|
||||
const scope = self as unknown as DedicatedWorkerGlobalScope;
|
||||
|
||||
function send(message: FromSandbox): void {
|
||||
scope.postMessage(message);
|
||||
}
|
||||
|
||||
/** Host calls waiting on the tab, by id. */
|
||||
const pendingHostCalls = new Map<number, (reply: string | Error) => void>();
|
||||
let nextHostCallId = 1;
|
||||
|
||||
const host = new PluginSandboxHost(
|
||||
(envelope) =>
|
||||
new Promise<string>((resolve, reject) => {
|
||||
const id = nextHostCallId++;
|
||||
pendingHostCalls.set(id, (reply) => (reply instanceof Error ? reject(reply) : resolve(reply)));
|
||||
send({ type: "host_call", id, envelope });
|
||||
}),
|
||||
(log) => send({ type: "log", ...log }),
|
||||
);
|
||||
|
||||
async function handle(message: ToSandbox): Promise<void> {
|
||||
if (message.type === "host_result") {
|
||||
const settle = pendingHostCalls.get(message.id);
|
||||
pendingHostCalls.delete(message.id);
|
||||
settle?.(message.error != null ? new Error(message.error) : (message.reply ?? "{}"));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
switch (message.type) {
|
||||
case "load":
|
||||
send({
|
||||
type: "result",
|
||||
id: message.id,
|
||||
result: await host.load(message.pluginRefId, message.source),
|
||||
});
|
||||
return;
|
||||
case "unload":
|
||||
host.unload(message.pluginRefId);
|
||||
send({ type: "result", id: message.id, result: null });
|
||||
return;
|
||||
case "dispatch":
|
||||
send({
|
||||
type: "result",
|
||||
id: message.id,
|
||||
result: await host.dispatch(message.pluginRefId, message.envelope),
|
||||
});
|
||||
return;
|
||||
}
|
||||
} catch (err) {
|
||||
send({
|
||||
type: "error",
|
||||
id: message.id,
|
||||
message: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
scope.onmessage = (e: MessageEvent<ToSandbox>) => void handle(e.data);
|
||||
Reference in New Issue
Block a user