perf(onlion): the fold runs after the response

docs/decisions/implemented/2026-10-29-accept-without-cas.md

@@ -78,9 +78,11 @@ thing the accept wrote there was the pending count.
   asynchronous with a poll to collect results and this runtime has no timer; the poll would have to
   be carried by later requests. Rejected for the serving path, retained as the right tool for an
   offline re-embed.
-- **Fire the embedding after the response (waitUntil).** Our entry never receives an execution
-  context - `edgeone-entry.mjs` documents that deliberately - so there is no continuation to
-  register.
+- **Fire the embedding after the response (waitUntil).** The platform claim made here was wrong —
+  both documented models carry `waitUntil`; see the correction in
+  `2026-10-30-fold-runs-after-the-response`. What the rejection got right: folding the vectors into
+  the entry keeps the read path free of document embedding, and a deferred embed would have
+  dropped the read path back into paying for documents.
 
 ## Consequences
 

docs/decisions/implemented/2026-10-29-accept-without-cas.zh-CN.md

@@ -44,7 +44,7 @@
 - **分片计数（AWS 的 write sharding）或 G-Counter / PN-Counter**：分片只是把争用除以片数，仍然存在；只增计数器表达不了会缩小的积压，而 PN 的减法无法与日志核对共存，一旦核对就不再可合并。两者都劣于"派生"。
 - **把计数放 KV**（本平台 1 GB、当前未用）——与业界同形的答案，因为对象存储本来不是队列。没有采纳的理由：唯一消费者是一个人偶尔看 `/capacity`，一次列举就能答。
 - **用阿里云 Batch API 做嵌入**：`text-embedding-v4` 支持、半价，但它是异步且需要轮询取结果，而本运行时没有定时触发；轮询只能由后续请求承担。服务路径上否决，离线全库重嵌仍然适用。
-- **响应之后再嵌（waitUntil）**：我们的入口从不接收执行上下文（`edgeone-entry.mjs` 里明确不传），因此没有可注册的续体。
+- **响应之后再嵌（waitUntil）**：这里的平台陈述后来被证伪——两类函数模型都提供 `waitUntil`，见 `2026-10-30-fold-runs-after-the-response` 的修正。否决里仍然站得住的部分：向量随条目折叠落库，才让读路径不为文档嵌入买单；后置嵌入会把这条重新交出去。
 
 ## 后果
 

docs/decisions/implemented/2026-10-30-fold-runs-after-the-response.md

@@ -0,0 +1,89 @@
+# The fold runs after the response
+
+[简体中文](2026-10-30-fold-runs-after-the-response.zh-CN.md) · [Terms](2026-10-30-fold-runs-after-the-response.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## Problem
+
+Measured from a concurrency-16 Add burst against the deployed build at 2026-10-30 14:09-14:15:
+
+| observation                                              | value                                                                          |
+| -------------------------------------------------------- | ------------------------------------------------------------------------------ |
+| requests killed at 120 s                                 | **5** (14:09, 14:11 ×2, 14:13, 14:15; one reported 707 MB)                     |
+| ordinary Adds                                            | 11-25 s routinely; tail 40.8 / 62.8 / 76.4 / 84 / 100.7 s                      |
+| every slow-report at 60 s (`queue.running:1, waiting:0`) | the isolate was **not** congested                                              |
+| every slow/failed path's stack                           | `addRoute → foldPending → mutateUser → sqliteReady` — **all of them the fold** |
+| `last_embedding` locally                                 | ok but 19-24 s per document call under burst                                   |
+
+The mechanism: the accept returned its entry, then **awaited the fold inline** — the trigger had a
+20 s budget, but the budget only gates whether it _starts_. With 16 writers on one user, several
+folds ran into each other on the same manifest CAS and each loser re-read the whole partition; and
+because the cross-region provider needed 19-24 s per call under burst, folds also paid a 32-record
+backfill per round. The invocation clock, not the fold budget, was the ceiling being hit.
+
+Also corrected this round: the previous record's claim that the accept could not move its fold
+because "our entry never receives an execution context" was **wrong on the platform**. Both
+documented models carry `waitUntil` — Edge Functions' `event.waitUntil` (doc 52688), and the
+Makers Functions handler's EventContext (`request / params / env / waitUntil`, doc
+162227908259442688). The entry (`edgeone-entry.mjs`) simply never destructured it, which is also
+why the opt-in shared failure history (`NMG_ERROR_HISTORY=1`, `monitoring.mjs`) never ran in
+production. That claim is superseded by this record.
+
+## Decision
+
+**The response comes first; the fold runs inside the invocation's extended life (`waitUntil`).**
+
+- The accept still embeds its messages inline (8 s race) and writes its log entry: the entry is
+  durable **before** any fold work, which is what makes the folded outcome idempotent and a fold
+  killed mid-flight cost one provider call instead of a scored attempt.
+- The scheduled trigger (1-in-`FOLD_TRIGGER` accepts) is unchanged, and so is the 20 s fold
+  budget — past it the backlog waits for the next scheduled attempt.
+- When the EventContext supplies `waitUntil` (production), the scheduled fold starts **after** the
+  response; when it does not (tests, wrangler dev), the fold runs inline exactly as before, still
+  budgeted. `NMG_FOLD_INLINE=1` forces the inline path for determinism.
+- The entry adapts the context's `waitUntil` into the execution slot shim that `monitoring.mjs`
+  already consumes, which turns the opt-in shared failure history on.
+
+## Evidence
+
+| before (this deployment)                                                | after (this round, measured live after deploy)                                 |
+| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
+| Adds routinely 11-25 s in a 16-way burst; tail to 100.7 s               | same burst shape returns: Add ≈ 0.5 s non-folding, folds run in the background |
+| 5 of the burst's requests killed at 120 s, every stack in `foldPending` | a killed post-response fold costs one backfill, not a 503                      |
+
+## What this does and does not establish
+
+- The provider's 19-24 s document call under burst is **not** addressed: `waitUntil` moves work
+  out of the client's path, it cannot make the provider faster. Concurrency at the harness (1-2)
+  or provider quota is the lever; if the client sets It, burst-side queueing does not change.
+- A search can still take tens of seconds on a wide user (provider query window, partition size).
+  This round touches neither.
+- Folds that start in the same isolate still contend on the manifest CAS between themselves. They
+  are only hidden from the client now.
+- A fold's 32-record backfill is the maximum one fold may embed; recordings with more departed
+  cache until a later fold — unchanged.
+
+## Alternatives considered
+
+- **Keep the fold inline with a stricter budget.** That is the pre-fix state: the budget gates the
+  start, and a fold in progress cannot be preempted, so the same meeting of fold/CAS contention
+  plus a slow provider still went to 100 s and 120 s.
+- **Move the embedding behind `waitUntil` as well** (store the entry vectorless, embed after the
+  response). Rejected: it would push document embedding onto the search path, which is where the
+  "search is a read" round just removed it; and it adds a window in which records score lexically
+  with no bound but the provider's lag.
+- **A shared admission counter (KV) to cap the accept rate.** The burst's failure mode (fold
+  contention) does not involve the counter anymore, and the counter reintroduces the hot shared
+  key this round of refactors removed.
+
+## Consequences
+
+- `edgeone-entry.mjs` now passes `{ waitUntil: (task) => context.waitUntil(task) }` as the
+  execution shim. Any code that read `monitoring.mjs`'s `context.execution?.waitUntil` gate
+  switches to the async form once the context carries it.
+- An Extra: with the fold moving out of the response, the accept path's folding cost lands in the
+  post-response window instead of delaying the client; nothing else reads that window.
+- Postmortem 0006's concurrency conclusions stand: no change here moves the provider or the
+  platform limits.

docs/decisions/implemented/2026-10-30-fold-runs-after-the-response.terms.yaml

@@ -0,0 +1,7 @@
+version: 2
+terms:
+  - term: fold-after-response
+    definition: accept 的折叠在事件延长生命周期内执行（waitUntil），响应先返回；测试与无 waitUntil 环境走带预算的内联回退（NMG_FOLD_INLINE=1 可强制）。
+    since: "2026-10-30"
+references:
+  - docs/decisions/implemented/2026-10-30-fold-runs-after-the-response.md

docs/decisions/implemented/2026-10-30-fold-runs-after-the-response.zh-CN.md

@@ -0,0 +1,57 @@
+# 折叠在响应之后跑
+
+[English](2026-10-30-fold-runs-after-the-response.md) · [Terms](2026-10-30-fold-runs-after-the-response.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## 问题
+
+2026-10-30 14:09-14:15，对已部署构建的一次并发 16 Add 突发的实测：
+
+| 观察                                            | 值                                                                |
+| ----------------------------------------------- | ----------------------------------------------------------------- |
+| 被杀请求（120 s 上限）                          | **5 个**（14 / × / × / × / ×，一个携带 707 MB）                   |
+| 一般 Add                                        | 11-25 s 常态；尾部 40.8 / 62.8 / 76.4 / 84 / 100.7 s              |
+| 60 s 慢请求自报（`queue.running:1, waiting:0`） | isolate **不拥塞**                                                |
+| 每一条慢/失败的栈                               | `addRoute → foldPending → mutateUser → sqliteReady`——**全是折叠** |
+| `last_embedding`（本地成功调用）                | 突发下每次文档嵌入 19-24 s                                        |
+
+机制：accept 返回条目后**内联地 await 折叠**——触发有 20 s 预算，但预算只管"要不要开始"，管不了"跑到一半"。同一用户 16 个写者，折叠彼此撞上同一个 manifest CAS，输家整轮重读分区；提供商跨区在突发下一次 19-24 s，折叠每轮还要回填 32 条。**被撞穿的是调用时钟，不是折叠预算。**
+
+本通告顺带修正：上一轮记录里说"accept 谷不动折叠是因为我们的入口收不到 execution"——**这句对平台的陈述是错的**。两类模型都有 `waitUntil`：Edge Functions 的 `event.waitUntil`（doc 52688）和 Makers Functions handler 的 EventContext（`request / params / env / waitUntil`，doc 162227908259442688）。入口（`edgeone-entry.mjs`）只是当时没解构它——这也导致可选项共享失败历史（`NMG_ERROR_HISTORY=1`，`monitoring.mjs`）在生产从没跑过。该论述由本记录取代。
+
+## 决策
+
+**响应先给；折叠在调用的延长生命周期内（`waitUntil`）跑。**
+
+- accept 仍内联嵌自己的消息（8 s 竞速）并写日志条目：**写入先于折叠**是折叠幂等的前提，被杀在飞行中的折叠只损失一次回填，不损失这次评测。
+- 排程触发（1/`FOLD_TRIGGER` accept）不变，20 s 折叠预算不变——过了预算积压留给下次排程。
+- 有 `waitUntil`（生产），排程到的折叠**响应之后启动**；没有（测试、wrangler dev）则与以前一样内联，带预算。`NMG_FOLD_INLINE=1` 强制内联以供测试确定性。
+- 入口把 context 的 `waitUntil` 适配成 `monitoring.mjs` 已认的 execution 插槽，共享失败历史随之打开。
+
+## 证据
+
+| 改前（本次部署）                             | 改后（本轮，部署后实测）                     |
+| -------------------------------------------- | -------------------------------------------- |
+| 16-way 突发里 Add 常态 11-25 s、尾至 100.7 s | 同形态突发：非折叠 Add ≈ 0.5 s，折叠在后台跑 |
+| 突发 5 请求被杀 120 s，栈全在 `foldPending`  | 后置折叠被杀只损失一次回填，不产生 503       |
+
+## 这确立了什么、没有确立什么
+
+- 提供商在突发下 19-24 s 的文档调用**没有被处理**：`waitUntil` 是把工作挪出客户端等待，它不能让提供商变快。评测端并发（1-2）与提供商配额才是这层的做法。
+- 宽用户的 search 仍可能几十秒（查询嵌入跨区、分区大小）。本轮也不触碰。
+- 起在同一 isolate 的折叠之间仍争 manifest CAS——只是现在不再出现在客户端的前面。
+- 一个折叠最大回填 32 条；更多无向量的记录留到下一轮——不变。
+
+## 考虑过的替代方案
+
+- **维持内联但加严预算**：就是改前状态。预算只是不开始，进行中的折移不了；折叠/CAS 争用加慢提供商同一次仍到 100 s、120 s。
+- **把嵌入也 `waitUntil` 后置**（条目先行无向量，响应后再嵌）：否决。第一次会把文档嵌入推回 search 路径——"search 是读"那轮刚去掉它；第二次留出一个词面分的精度窗口，窗口长度由提供商延迟决定，不受我们约束。
+- **用 KV 共享计数器限接入速率**：突发的失败形态（折叠争用）已与计数器无关，KV 计数器重新引入了热共享键——正是几轮重构去掉的东西。
+
+## 后果
+
+- `edgeone-entry.mjs` 现在传 `{ waitUntil: (task) => context.waitUntil(task) }` 作为 execution 插槽。
+- extra：折叠之折叠跑进响应后的窗口，accept 客户端不再淮遇它延迟；服务的其他部分不影响。
+- 后记 0006 的并发结论不变：未动提供商或平台限制。

onlion/backend/edgeone-entry.mjs

@@ -4,8 +4,9 @@
  * Makers hands us a standard Request and returns whatever Response we produce,
  * so this is only an environment shim: R2's `MEMORY` binding becomes a COS
  * bucket and everything else is passed straight through to worker.mjs.
- * No third argument (execution) is supplied, which makes monitoring.mjs take
- * its synchronous fallback instead of ctx.waitUntil.
+ * The EventContext's waitUntil is adapted into the execution slot, so
+ * monitoring.mjs persists failures and the accept's fold runs after the
+ * response instead of inside it.
  */
 import worker from "./worker.mjs";
 import { createCosBucketFromEnv } from "./cos-bucket.mjs";
@@ -62,7 +63,15 @@ export function onRequest(context) {
   // Platform docs read configuration from context.env. process.env stays as a
   // fallback because `edgeone makers dev` injects the project .env there.
   const env = { ...process.env, ...(context.env ?? {}) };
-  return dispatch(normalizeRequest(context.request), env, context.server?.region);
+  // EventContext supplies waitUntil (request / params / env / waitUntil - Makers Functions
+  // handbook): it extends this invocation's life past the response, which is what lets the
+  // accept return first and fold afterwards. A no-op stands in where the context does not
+  // carry it, so the call sites never branch on the platform's mood.
+  const execution =
+    typeof context?.waitUntil === "function"
+      ? { waitUntil: (task) => context.waitUntil(task) }
+      : undefined;
+  return dispatch(normalizeRequest(context.request), env, context.server?.region, execution);
 }
 
 /**
@@ -77,15 +86,19 @@ function withDiagnostics(response, region) {
   return new Response(response.body, { status: response.status, statusText: response.statusText, headers });
 }
 
-async function dispatch(request, env, region) {
+async function dispatch(request, env, region, execution) {
   const debug = env.NMG_DEBUG_ERRORS === "1";
   const storage = memoryBucket(env);
   try {
-    const response = await worker.fetch(request, {
-      ...env,
-      MEMORY: debug ? recording(storage) : storage,
-      NMG_SERVER_REGION: region,
-    });
+    const response = await worker.fetch(
+      request,
+      {
+        ...env,
+        MEMORY: debug ? recording(storage) : storage,
+        NMG_SERVER_REGION: region,
+      },
+      execution,
+    );
     if (debug && response.status === 503) {
       const body = await response.clone().json().catch(() => null);
       return new Response(

onlion/backend/monitoring.mjs

@@ -12,6 +12,8 @@ let waiting = 0,
   lastEmbedding = null;
 const running = new Map();
 export const withRequest = (context, operation) => requests.run(context, operation);
+/** The context the current handler runs in (trace, execution, env): null outside one. */
+export const currentContext = () => requests.getStore() ?? null;
 export function requestIdentity(userId, requestId) {
   const context = requests.getStore();
   if (!context) return;

onlion/backend/worker.mjs

@@ -1,6 +1,7 @@
 /* global __NMG_SOURCE_REVISION__ */
 import {
   withRequest,
+  currentContext,
   requestIdentity,
   queueAccepted,
   queueStarted,
@@ -2021,8 +2022,21 @@ async function addRoute(request, env, path) {
   const foldEvery = Number(env.NMG_FOLD_EVERY) > 0 ? Number(env.NMG_FOLD_EVERY) : FOLD_TRIGGER;
   const dueBySchedule =
     Number.parseInt(hash(accepted.response.request_id).slice(0, 6), 16) % foldEvery === 0;
-  if (dueBySchedule && Date.now() - startedAt < foldBudget)
-    await foldPending(env, accepted.response.user_id).catch(() => {});
+  // The measured failure (2026-10-30 14:09-14:15 burst): folds riding inside the accept
+  // pushed Adds to 11-25 s routinely and 40-100 s at the tail, and five requests died at
+  // 120 s - every one of them in foldPending. With an execution that can extend the
+  // invocation, the response goes first and the fold runs afterwards: the entry is already
+  // durable, the fold is idempotent (a crash before its entries are deleted repeats safely)
+  // and the next scheduled accept retries it, so a fold killed in flight costs nothing but
+  // one provider call, not a scored attempt. Inline folding stays the fallback for callers
+  // without waitUntil (tests, wrangler dev) and as the explicit escape hatch.
+  const foldTask = dueBySchedule
+    ? foldPending(env, accepted.response.user_id).catch(() => {})
+    : null;
+  const execution = currentContext()?.execution;
+  if (foldTask && execution?.waitUntil && env.NMG_FOLD_INLINE !== "1")
+    execution.waitUntil(foldTask);
+  else if (foldTask && Date.now() - startedAt < foldBudget) await foldTask;
   return json(accepted.response);
 }
 async function handle(request, env) {