perf(onlion): stop paying for SQLite in requests that never touch it

docs/decisions/implemented/2026-10-30-defer-wasm-load.md

@@ -0,0 +1,72 @@
+# SQLite is loaded by the requests that need it
+
+[简体中文](2026-10-30-defer-wasm-load.zh-CN.md) · [Terms](2026-10-30-defer-wasm-load.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## Problem
+
+Measured on the live endpoint before this change: **twelve sequential `/health` calls took
+3.6-6.5 s each**. Every one of them is a cold start, because this platform starts a **fresh isolate
+per request** (every log line carries `instance.seq = 1` next to a container uptime in the
+thousands of seconds). So that figure was paid per request, not once per container, and it was
+spent on SQLite that `/health` does not use:
+
+1. `healthResponse` called `sqliteReady()` before answering, purely to report
+   `sqlite.version.libVersion`.
+2. The EdgeOne build inlined the wasm as base64 and emitted
+   `export default new WebAssembly.Module(Buffer.from(<base64>, "base64"))` **at module scope** -
+   the plugin's comment said "lazily built", the emitted code was eager. A 1.6 MiB base64 decode
+   plus a wasm compile therefore ran before any handler.
+
+The harness preflight is `/health`. It was the most expensive route in the service.
+
+## Decision
+
+- The build emits a **factory**: `encodedBytes` stays a string literal (cheap), and
+  `compiledWasm()` decodes and compiles on first call.
+- `worker.mjs` accepts either shape - a factory, or workerd's precompiled module in the wrangler
+  build - so the two build paths stay one code path.
+- `/health` does not load SQLite; it answers "deferred until a request needs it" instead of a
+  version string it did not read.
+- The always-on request line carries `sqlite_loaded`, so the claim is falsifiable: `false` means
+  that request finished without decoding or compiling wasm at all.
+
+## Evidence
+
+| before                                                                                                                 | after                                                                                               |
+| ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
+| `/health` x12 on live: 3.617 / 3.839 / 3.772 / 6.482 / 4.301 / 4.729 / 5.861 / 6.039 / 5.143 / 3.745 / 4.249 / 3.604 s | to be measured after deploy; `sqlite_loaded: false` on the metadata routes is asserted in the suite |
+| suite: metadata routes logged `sqlite_loaded: false`                                                                   | same assertion, now also checkable in the platform log per request                                  |
+
+## What this does and does not establish
+
+- It removes the wasm decode/compile from routes that never touch a database. It does **not** make
+  a database request cheaper: `/add` and `/search` still load SQLite, and on a fresh-isolate-per-
+  request platform they still pay for it on every call. That is the next lever, and it is a
+  different design (a runtime that keeps the isolate alive, or no wasm at all).
+- The 3.6-6.5 s figure includes container start and network; only the delta after deploy is
+  attributable to this change.
+- Nothing here addresses the 53 ms / 54.48 MB fast failures: those invocations never reached the
+  handler.
+
+## Alternatives considered
+
+- **Keep the eager module and only drop `sqliteReady()` from `/health`.** Half the cost: the
+  base64 decode and compile still run at module scope, so the route would stay slow.
+- **Ship the wasm as a separate file and read it lazily.** The EdgeOne build deliberately inlines
+  everything ("removes every filesystem-path assumption"): the function is a single module in
+  `/var/user`, and file access there is not a contract. Keeping the bytes inline but decoded late
+  gets the same win without reintroducing a path dependency.
+- **Cache the compiled module across requests.** Not possible: the isolate is per request, so a
+  module-level cache is per request by definition.
+
+## Consequences
+
+- `/health` no longer reports `sqlite.version.libVersion`; it reports that the load is deferred.
+  Any client that parsed that field must read the version from `/admin/status` instead - the
+  harness only reads the status code.
+- `sqlite_loaded` is one boolean on the request line; no extra allocation, no memory sampling.
+- The wrangler build is untouched in behaviour: workerd still compiles the module and
+  `worker.mjs` uses the object directly.

docs/decisions/implemented/2026-10-30-defer-wasm-load.terms.yaml

@@ -0,0 +1,4 @@
+version: 2
+terms: []
+references:
+  - docs/decisions/implemented/2026-10-30-defer-wasm-load.md

docs/decisions/implemented/2026-10-30-defer-wasm-load.zh-CN.md

@@ -0,0 +1,47 @@
+# SQLite 只由需要它的请求加载
+
+[English](2026-10-30-defer-wasm-load.md) · [Terms](2026-10-30-defer-wasm-load.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## 问题
+
+改动前在线上实测：**连续 12 次 `/health` 各耗 3.6–6.5 秒**。每一次都是冷启动——本平台**每请求一个全新 isolate**（每行日志 `instance.seq = 1`，而容器 `uptime_s` 是几千秒）。所以这个开销是**按请求**付的，不是按容器付一次；而且付在了 `/health` 根本用不到的 SQLite 上：
+
+1. `healthResponse` 在应答前调用 `sqliteReady()`，只为报告 `sqlite.version.libVersion`；
+2. EdgeOne 构建把 wasm 以 base64 内联，并在**模块顶层**发出 `export default new WebAssembly.Module(Buffer.from(<base64>, "base64"))`——插件注释写着"lazy"，实际代码是急切的。于是 1.6 MiB 的 base64 解码加 wasm 编译在任何 handler 之前就跑完了。
+
+评测预检恰好打的是 `/health`。它成了服务里最贵的一条路由。
+
+## 决策
+
+- 构建改为发出**工厂**：`encodedBytes` 保持字符串字面量（便宜），`compiledWasm()` 在首次调用时才解码与编译。
+- `worker.mjs` 接受两种形态——工厂，或 wrangler 构建里 workerd 预编译好的模块——两条构建路径共用一份代码。
+- `/health` 不再加载 SQLite，回答"deferred until a request needs it"，而不是一个它没读过的版本号。
+- 常开请求行带 `sqlite_loaded`，让这个结论可被证伪：`false` 表示该请求全程没有解码或编译 wasm。
+
+## 证据
+
+| 改前                                                                                                                 | 改后                                                         |
+| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
+| 线上 `/health` ×12：3.617 / 3.839 / 3.772 / 6.482 / 4.301 / 4.729 / 5.861 / 6.039 / 5.143 / 3.745 / 4.249 / 3.604 秒 | 部署后实测；元数据路由 `sqlite_loaded: false` 已在套件中断言 |
+| 套件里元数据路由记录 `sqlite_loaded: false`                                                                          | 同一断言，现在还能按请求在平台日志里核对                     |
+
+## 这确立了什么、没有确立什么
+
+- 它把 wasm 解码/编译从"不碰数据库的路由"上移除。它**没有**让数据库请求变便宜：`/add` 与 `/search` 仍加载 SQLite，在"每请求一个新 isolate"的平台上仍然每次都要付。那是下一个杠杆，而且属于另一种设计（能复用 isolate 的运行时，或者干脆不用 wasm）。
+- 3.6–6.5 秒里包含容器启动与网络；部署后只有**差值**能归因于本次改动。
+- 它不处理 53 ms / 54.48 MB 的快速失败：那些调用从未进入 handler。
+
+## 考虑过的替代方案
+
+- **保留急切模块，只从 `/health` 去掉 `sqliteReady()`**：只省一半——base64 解码与编译仍在模块顶层，路由照样慢。
+- **把 wasm 作为独立文件、按需读取**：EdgeOne 构建刻意内联一切（"removes every filesystem-path assumption"）；函数是 `/var/user` 下的单个模块，那里的文件访问不是契约。字节留内联、解码推迟，能拿到同样的收益而不重新引入路径依赖。
+- **跨请求缓存编译好的模块**：做不到——isolate 每请求一个，模块级缓存天然就是"每请求一份"。
+
+## 后果
+
+- `/health` 不再报告 `sqlite.version.libVersion`，改为报告"延迟加载"。解析该字段的客户端应改从 `/admin/status` 读版本——评测端只用状态码。
+- `sqlite_loaded` 是请求行上一个布尔值，无额外分配、无内存采样。
+- wrangler 构建行为不变：workerd 依旧编译模块，`worker.mjs` 直接使用该对象。

onlion/backend/sqlite.mjs

@@ -16,6 +16,11 @@ function noteImage() {
   highWater.databases = Math.max(highWater.databases, databases);
   highWater.imageBytes = Math.max(highWater.imageBytes, imageBytes);
 }
+/** Whether this invocation has loaded the SQLite module at all. Reported per request so that
+ *  "the deferred load works" is a measurement, not a claim. */
+export function sqliteIsReady() {
+  return active;
+}
 /** What the memory trace reports: the live images in this request, the instance high-water
  *  mark, and the WASM linear memory size (which only ever grows). */
 export function sqliteStats() {

onlion/backend/worker.mjs

@@ -27,7 +27,13 @@ import {
 } from "./diagnostics.mjs";
 import init from "../sqlite-wasm/worker-loader.mjs";
 import compiledSqlite from "../sqlite-wasm/sqlite3.wasm";
-import { configure, DatabaseSync, withDatabases, sqliteStats } from "./sqlite.mjs";
+import {
+  configure,
+  DatabaseSync,
+  withDatabases,
+  sqliteStats,
+  sqliteIsReady,
+} from "./sqlite.mjs";
 import { memRequestStarted, memLog, memFields } from "./mem-trace.mjs";
 import { AmlMemoryService, AML_SEARCH_PROFILE, CORE_SOURCE } from "./memory-service.ts";
 import { memoryEmbeddingText } from "../../src/core/store/search-ranking.ts";
@@ -163,10 +169,15 @@ function createContentView() {
 }
 async function sqliteReady() {
   if (ready) return ready;
+  // Two build shapes: workerd (wrangler) hands us a compiled module object, while the EdgeOne
+  // build inlines the bytes behind a factory so that a request which never touches SQLite does
+  // not pay for decoding and compiling them. This platform starts a fresh isolate per request,
+  // so that saving is per request, not per container.
+  const compiled = typeof compiledSqlite === "function" ? compiledSqlite() : compiledSqlite;
   const instance = await init({
     instantiateWasm(imports, receive) {
-      const instance = new WebAssembly.Instance(compiledSqlite, imports);
-      receive(instance, compiledSqlite);
+      const instance = new WebAssembly.Instance(compiled, imports);
+      receive(instance, compiled);
       return instance.exports;
     },
     print: () => {},
@@ -1607,13 +1618,15 @@ async function healthResponse(env, path) {
   if (path !== "/health") return undefined;
   if (!env.MEMORY)
     throw new HttpError(503, "storage_unavailable", "Persistent storage binding missing");
-  const sqlite = await sqliteReady();
+  // Deliberately no sqliteReady(): /health is the harness preflight, it is hit first, and it does
+  // not read a database. Loading SQLite here made the cheapest route pay a wasm decode and
+  // compile measured at 3.6-6.5 s per invocation on this platform.
   await env.MEMORY.head("health/probe");
   return json({
     status: "ok",
     service: "NodeMemoryGraph",
     storage: "R2",
-    sqlite: sqlite.version.libVersion,
+    sqlite: "deferred until a request needs it",
     retrieval: env.NMG_EMBED_PROVIDER || env.NMG_EMBED_BASE_URL ? "hybrid" : "lexical",
     max_user_database_bytes: MAX_DATABASE_BYTES,
     storage_format: "nmg-shards-v3",
@@ -2235,6 +2248,9 @@ export default {
             elapsed_ms: Date.now() - startedAt,
             seq: context.seq,
             execution: context.execution?.waitUntil ? "waituntil" : "inline",
+            // Cheap, and the only way to prove the deferred wasm load works: false means this
+            // request finished without decoding or compiling SQLite at all.
+            sqlite_loaded: sqliteIsReady(),
           }),
         );
         memLog("exit", route, {

onlion/scripts/build-edgeone.mjs

@@ -41,16 +41,31 @@ const sourceRevision = revision + (dirty ? "+dirty" : "");
 fs.rmSync(outDir, { recursive: true, force: true });
 fs.mkdirSync(path.join(outDir, "cloud-functions"), { recursive: true });
 
-/** Replaces `import wasm from './x.wasm'` with a lazily built WebAssembly.Module. */
+/** Replaces `import wasm from './x.wasm'` with a lazily built WebAssembly.Module.
+ *
+ *  It has to be lazy, and the first version was not: it emitted
+ *  `export default new WebAssembly.Module(Buffer.from(<base64>, "base64"))` at module scope, so
+ *  every cold start paid a 1.6 MiB base64 decode plus a wasm compile *before* the handler ran.
+ *  This platform starts a fresh isolate per request, so that cost was charged to every request -
+ *  measured as 3.6-6.5 s for `/health`, which never needs SQLite at all. The bytes and the
+ *  compile now happen on first use, behind a factory; `worker.mjs` accepts either shape so the
+ *  wrangler build (where workerd compiles the module) keeps working unchanged. */
 const inlineWasm = {
   name: "inline-wasm",
   setup(pluginBuild) {
     pluginBuild.onLoad({ filter: /\.wasm$/ }, async (args) => {
       const bytes = fs.readFileSync(args.path).toString("base64");
-      const module = `Buffer.from(${JSON.stringify(bytes)}, "base64")`;
+      const encoded = JSON.stringify(bytes);
       return {
         loader: "js",
-        contents: `export default new WebAssembly.Module(${module});`,
+        contents: [
+          `let compiled;`,
+          `export const encodedBytes = ${encoded};`,
+          `export default function compiledWasm() {`,
+          `  compiled ??= new WebAssembly.Module(Buffer.from(encodedBytes, "base64"));`,
+          `  return compiled;`,
+          `}`,
+        ].join("\n"),
         watchFiles: [args.path],
       };
     });