diff --git a/docs/decisions/implemented/2026-10-30-defer-wasm-load.md b/docs/decisions/implemented/2026-10-30-defer-wasm-load.md new file mode 100644 index 00000000..4252f998 --- /dev/null +++ b/docs/decisions/implemented/2026-10-30-defer-wasm-load.md @@ -0,0 +1,72 @@ +# SQLite is loaded by the requests that need it + +[简体中文](2026-10-30-defer-wasm-load.zh-CN.md) · [Terms](2026-10-30-defer-wasm-load.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## Problem + +Measured on the live endpoint before this change: **twelve sequential `/health` calls took +3.6-6.5 s each**. Every one of them is a cold start, because this platform starts a **fresh isolate +per request** (every log line carries `instance.seq = 1` next to a container uptime in the +thousands of seconds). So that figure was paid per request, not once per container, and it was +spent on SQLite that `/health` does not use: + +1. `healthResponse` called `sqliteReady()` before answering, purely to report + `sqlite.version.libVersion`. +2. The EdgeOne build inlined the wasm as base64 and emitted + `export default new WebAssembly.Module(Buffer.from(, "base64"))` **at module scope** - + the plugin's comment said "lazily built", the emitted code was eager. A 1.6 MiB base64 decode + plus a wasm compile therefore ran before any handler. + +The harness preflight is `/health`. It was the most expensive route in the service. + +## Decision + +- The build emits a **factory**: `encodedBytes` stays a string literal (cheap), and + `compiledWasm()` decodes and compiles on first call. +- `worker.mjs` accepts either shape - a factory, or workerd's precompiled module in the wrangler + build - so the two build paths stay one code path. +- `/health` does not load SQLite; it answers "deferred until a request needs it" instead of a + version string it did not read. +- The always-on request line carries `sqlite_loaded`, so the claim is falsifiable: `false` means + that request finished without decoding or compiling wasm at all. + +## Evidence + +| before | after | +| ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | +| `/health` x12 on live: 3.617 / 3.839 / 3.772 / 6.482 / 4.301 / 4.729 / 5.861 / 6.039 / 5.143 / 3.745 / 4.249 / 3.604 s | to be measured after deploy; `sqlite_loaded: false` on the metadata routes is asserted in the suite | +| suite: metadata routes logged `sqlite_loaded: false` | same assertion, now also checkable in the platform log per request | + +## What this does and does not establish + +- It removes the wasm decode/compile from routes that never touch a database. It does **not** make + a database request cheaper: `/add` and `/search` still load SQLite, and on a fresh-isolate-per- + request platform they still pay for it on every call. That is the next lever, and it is a + different design (a runtime that keeps the isolate alive, or no wasm at all). +- The 3.6-6.5 s figure includes container start and network; only the delta after deploy is + attributable to this change. +- Nothing here addresses the 53 ms / 54.48 MB fast failures: those invocations never reached the + handler. + +## Alternatives considered + +- **Keep the eager module and only drop `sqliteReady()` from `/health`.** Half the cost: the + base64 decode and compile still run at module scope, so the route would stay slow. +- **Ship the wasm as a separate file and read it lazily.** The EdgeOne build deliberately inlines + everything ("removes every filesystem-path assumption"): the function is a single module in + `/var/user`, and file access there is not a contract. Keeping the bytes inline but decoded late + gets the same win without reintroducing a path dependency. +- **Cache the compiled module across requests.** Not possible: the isolate is per request, so a + module-level cache is per request by definition. + +## Consequences + +- `/health` no longer reports `sqlite.version.libVersion`; it reports that the load is deferred. + Any client that parsed that field must read the version from `/admin/status` instead - the + harness only reads the status code. +- `sqlite_loaded` is one boolean on the request line; no extra allocation, no memory sampling. +- The wrangler build is untouched in behaviour: workerd still compiles the module and + `worker.mjs` uses the object directly. diff --git a/docs/decisions/implemented/2026-10-30-defer-wasm-load.terms.yaml b/docs/decisions/implemented/2026-10-30-defer-wasm-load.terms.yaml new file mode 100644 index 00000000..0a26a1ec --- /dev/null +++ b/docs/decisions/implemented/2026-10-30-defer-wasm-load.terms.yaml @@ -0,0 +1,4 @@ +version: 2 +terms: [] +references: + - docs/decisions/implemented/2026-10-30-defer-wasm-load.md diff --git a/docs/decisions/implemented/2026-10-30-defer-wasm-load.zh-CN.md b/docs/decisions/implemented/2026-10-30-defer-wasm-load.zh-CN.md new file mode 100644 index 00000000..38b82f8b --- /dev/null +++ b/docs/decisions/implemented/2026-10-30-defer-wasm-load.zh-CN.md @@ -0,0 +1,47 @@ +# SQLite 只由需要它的请求加载 + +[English](2026-10-30-defer-wasm-load.md) · [Terms](2026-10-30-defer-wasm-load.terms.yaml) + +**Status:** implemented +**Approved:** explicit + +## 问题 + +改动前在线上实测：**连续 12 次 `/health` 各耗 3.6–6.5 秒**。每一次都是冷启动——本平台**每请求一个全新 isolate**（每行日志 `instance.seq = 1`，而容器 `uptime_s` 是几千秒）。所以这个开销是**按请求**付的，不是按容器付一次；而且付在了 `/health` 根本用不到的 SQLite 上： + +1. `healthResponse` 在应答前调用 `sqliteReady()`，只为报告 `sqlite.version.libVersion`； +2. EdgeOne 构建把 wasm 以 base64 内联，并在**模块顶层**发出 `export default new WebAssembly.Module(Buffer.from(, "base64"))`——插件注释写着"lazy"，实际代码是急切的。于是 1.6 MiB 的 base64 解码加 wasm 编译在任何 handler 之前就跑完了。 + +评测预检恰好打的是 `/health`。它成了服务里最贵的一条路由。 + +## 决策 + +- 构建改为发出**工厂**：`encodedBytes` 保持字符串字面量（便宜），`compiledWasm()` 在首次调用时才解码与编译。 +- `worker.mjs` 接受两种形态——工厂，或 wrangler 构建里 workerd 预编译好的模块——两条构建路径共用一份代码。 +- `/health` 不再加载 SQLite，回答"deferred until a request needs it"，而不是一个它没读过的版本号。 +- 常开请求行带 `sqlite_loaded`，让这个结论可被证伪：`false` 表示该请求全程没有解码或编译 wasm。 + +## 证据 + +| 改前 | 改后 | +| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | +| 线上 `/health` ×12：3.617 / 3.839 / 3.772 / 6.482 / 4.301 / 4.729 / 5.861 / 6.039 / 5.143 / 3.745 / 4.249 / 3.604 秒 | 部署后实测；元数据路由 `sqlite_loaded: false` 已在套件中断言 | +| 套件里元数据路由记录 `sqlite_loaded: false` | 同一断言，现在还能按请求在平台日志里核对 | + +## 这确立了什么、没有确立什么 + +- 它把 wasm 解码/编译从"不碰数据库的路由"上移除。它**没有**让数据库请求变便宜：`/add` 与 `/search` 仍加载 SQLite，在"每请求一个新 isolate"的平台上仍然每次都要付。那是下一个杠杆，而且属于另一种设计（能复用 isolate 的运行时，或者干脆不用 wasm）。 +- 3.6–6.5 秒里包含容器启动与网络；部署后只有**差值**能归因于本次改动。 +- 它不处理 53 ms / 54.48 MB 的快速失败：那些调用从未进入 handler。 + +## 考虑过的替代方案 + +- **保留急切模块，只从 `/health` 去掉 `sqliteReady()`**：只省一半——base64 解码与编译仍在模块顶层，路由照样慢。 +- **把 wasm 作为独立文件、按需读取**：EdgeOne 构建刻意内联一切（"removes every filesystem-path assumption"）；函数是 `/var/user` 下的单个模块，那里的文件访问不是契约。字节留内联、解码推迟，能拿到同样的收益而不重新引入路径依赖。 +- **跨请求缓存编译好的模块**：做不到——isolate 每请求一个，模块级缓存天然就是"每请求一份"。 + +## 后果 + +- `/health` 不再报告 `sqlite.version.libVersion`，改为报告"延迟加载"。解析该字段的客户端应改从 `/admin/status` 读版本——评测端只用状态码。 +- `sqlite_loaded` 是请求行上一个布尔值，无额外分配、无内存采样。 +- wrangler 构建行为不变：workerd 依旧编译模块，`worker.mjs` 直接使用该对象。 diff --git a/onlion/backend/sqlite.mjs b/onlion/backend/sqlite.mjs index 8e122907..fceaeaf0 100644 --- a/onlion/backend/sqlite.mjs +++ b/onlion/backend/sqlite.mjs @@ -16,6 +16,11 @@ function noteImage() { highWater.databases = Math.max(highWater.databases, databases); highWater.imageBytes = Math.max(highWater.imageBytes, imageBytes); } +/** Whether this invocation has loaded the SQLite module at all. Reported per request so that + * "the deferred load works" is a measurement, not a claim. */ +export function sqliteIsReady() { + return active; +} /** What the memory trace reports: the live images in this request, the instance high-water * mark, and the WASM linear memory size (which only ever grows). */ export function sqliteStats() { diff --git a/onlion/backend/worker.mjs b/onlion/backend/worker.mjs index c9a6854d..d65857ad 100644 --- a/onlion/backend/worker.mjs +++ b/onlion/backend/worker.mjs @@ -27,7 +27,13 @@ import { } from "./diagnostics.mjs"; import init from "../sqlite-wasm/worker-loader.mjs"; import compiledSqlite from "../sqlite-wasm/sqlite3.wasm"; -import { configure, DatabaseSync, withDatabases, sqliteStats } from "./sqlite.mjs"; +import { + configure, + DatabaseSync, + withDatabases, + sqliteStats, + sqliteIsReady, +} from "./sqlite.mjs"; import { memRequestStarted, memLog, memFields } from "./mem-trace.mjs"; import { AmlMemoryService, AML_SEARCH_PROFILE, CORE_SOURCE } from "./memory-service.ts"; import { memoryEmbeddingText } from "../../src/core/store/search-ranking.ts"; @@ -163,10 +169,15 @@ function createContentView() { } async function sqliteReady() { if (ready) return ready; + // Two build shapes: workerd (wrangler) hands us a compiled module object, while the EdgeOne + // build inlines the bytes behind a factory so that a request which never touches SQLite does + // not pay for decoding and compiling them. This platform starts a fresh isolate per request, + // so that saving is per request, not per container. + const compiled = typeof compiledSqlite === "function" ? compiledSqlite() : compiledSqlite; const instance = await init({ instantiateWasm(imports, receive) { - const instance = new WebAssembly.Instance(compiledSqlite, imports); - receive(instance, compiledSqlite); + const instance = new WebAssembly.Instance(compiled, imports); + receive(instance, compiled); return instance.exports; }, print: () => {}, @@ -1607,13 +1618,15 @@ async function healthResponse(env, path) { if (path !== "/health") return undefined; if (!env.MEMORY) throw new HttpError(503, "storage_unavailable", "Persistent storage binding missing"); - const sqlite = await sqliteReady(); + // Deliberately no sqliteReady(): /health is the harness preflight, it is hit first, and it does + // not read a database. Loading SQLite here made the cheapest route pay a wasm decode and + // compile measured at 3.6-6.5 s per invocation on this platform. await env.MEMORY.head("health/probe"); return json({ status: "ok", service: "NodeMemoryGraph", storage: "R2", - sqlite: sqlite.version.libVersion, + sqlite: "deferred until a request needs it", retrieval: env.NMG_EMBED_PROVIDER || env.NMG_EMBED_BASE_URL ? "hybrid" : "lexical", max_user_database_bytes: MAX_DATABASE_BYTES, storage_format: "nmg-shards-v3", @@ -2235,6 +2248,9 @@ export default { elapsed_ms: Date.now() - startedAt, seq: context.seq, execution: context.execution?.waitUntil ? "waituntil" : "inline", + // Cheap, and the only way to prove the deferred wasm load works: false means this + // request finished without decoding or compiling SQLite at all. + sqlite_loaded: sqliteIsReady(), }), ); memLog("exit", route, { diff --git a/onlion/scripts/build-edgeone.mjs b/onlion/scripts/build-edgeone.mjs index b7ba0dd4..d07bd00f 100644 --- a/onlion/scripts/build-edgeone.mjs +++ b/onlion/scripts/build-edgeone.mjs @@ -41,16 +41,31 @@ const sourceRevision = revision + (dirty ? "+dirty" : ""); fs.rmSync(outDir, { recursive: true, force: true }); fs.mkdirSync(path.join(outDir, "cloud-functions"), { recursive: true }); -/** Replaces `import wasm from './x.wasm'` with a lazily built WebAssembly.Module. */ +/** Replaces `import wasm from './x.wasm'` with a lazily built WebAssembly.Module. + * + * It has to be lazy, and the first version was not: it emitted + * `export default new WebAssembly.Module(Buffer.from(, "base64"))` at module scope, so + * every cold start paid a 1.6 MiB base64 decode plus a wasm compile *before* the handler ran. + * This platform starts a fresh isolate per request, so that cost was charged to every request - + * measured as 3.6-6.5 s for `/health`, which never needs SQLite at all. The bytes and the + * compile now happen on first use, behind a factory; `worker.mjs` accepts either shape so the + * wrangler build (where workerd compiles the module) keeps working unchanged. */ const inlineWasm = { name: "inline-wasm", setup(pluginBuild) { pluginBuild.onLoad({ filter: /\.wasm$/ }, async (args) => { const bytes = fs.readFileSync(args.path).toString("base64"); - const module = `Buffer.from(${JSON.stringify(bytes)}, "base64")`; + const encoded = JSON.stringify(bytes); return { loader: "js", - contents: `export default new WebAssembly.Module(${module});`, + contents: [ + `let compiled;`, + `export const encodedBytes = ${encoded};`, + `export default function compiledWasm() {`, + ` compiled ??= new WebAssembly.Module(Buffer.from(encodedBytes, "base64"));`, + ` return compiled;`, + `}`, + ].join("\n"), watchFiles: [args.path], }; });