fix(onlion): the fold budget became a deadline; the request line records the method

docs/decisions/implemented/2026-10-30-fold-budget-is-a-deadline.md

@@ -0,0 +1,80 @@
+# The fold budget is a deadline; and this runtime has no waitUntil
+
+[简体中文](2026-10-30-fold-budget-is-a-deadline.zh-CN.md) · [Terms](2026-10-30-fold-budget-is-a-deadline.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## Problem
+
+Two facts, both from production log lines rather than from reading code.
+
+**One: the previous round's premise was wrong for this runtime.** The always-on request line
+reports `execution: "inline"` on every request since the stabilisation deploy — the adapter's
+`context.waitUntil` probe never found the hook. That is correct behaviour on a platform that does
+not supply it: the Makers **Node cloud function** context documents `uuid / params / env /
+clientIp / server{region,requestId,geo}` and nothing else. The `waitUntil` the earlier record cited
+belongs to **Pages Edge Functions** (`FetchEvent`) and to the Pages edge handler — a different
+runtime from the Node cloud function this project deploys. So "the fold runs after the response"
+never took effect in production; the entry in `2026-10-30-fold-runs-after-the-response` claimed
+otherwise and is superseded here. The 15:00 Add p50 of 1.02 s with a 6.5 s tail was **inline**
+folding at a 1-in-8 schedule, and the 16:59 rows show the same: `execution: "inline"`.
+
+**Two: the fold budget only gated the start.** `if (dueBySchedule && Date.now() - startedAt <
+foldBudget)` decided whether a fold _began_; a fold beginning at 19.9 s then ran a whole partition
+rewrite plus its 32-record backfill. That is the mechanism behind the 14:09-14:15 numbers (Adds 11-25
+s, tail 100.7 s, five kills at 120 s, every stack in `foldPending`) and it survived the previous two
+rounds because nothing bounded the fold after it started.
+
+A third, smaller finding from the same logs: two `/add` and `/search` requests answered **404 in
+24-47 ms on dead-cold isolates** (81 MB, `wasm_linear_mb: null` — SQLite was never touched). Our
+`/add` and `/search` accept POST only, so those were not work requests; the request line now
+records `method` so the next occurrence is identified instead of guessed.
+
+## Decision
+
+- **The fold takes a deadline and honours it.** `foldPending(env, userId, { deadline })` refuses to
+  start past it and, between applying entries, stops and leaves the remainder of the batch in the
+  log. The accept passes `startedAt + foldBudget`; `/admin/fold` passes its own 100 s.
+  Stopping is always safe: the entry is durable, folding is idempotent, and the next scheduled
+  accept retries it.
+- **No dependency on `waitUntil`.** The branch stays for a runtime that provides one (tests do),
+  but the shipped path is the bounded inline fold, which is the only shape this platform supports.
+- The request line gains `method`. Diagnostic only.
+
+## Evidence
+
+| before                                                                               | after                                                                                                                                                |
+| ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 14:09-14:15: Adds 11-25 s, tail 40-100 s, 5 kills at 120 s, all stacks `foldPending` | a fold that runs out of budget returns `{ folded, remaining: null, deferred: "budget" }` and the rest of the log waits for the next scheduled accept |
+| 15:00 p50 1.02 s / max 6.5 s: inline folding at 1-in-8, budget gating only the start | same schedule, now bounded by the same clock the caller measured                                                                                     |
+| no `method` in the request line; two 404s of unknown origin                          | `method` recorded, so a 404 either becomes explainable or stays provably non-mutating                                                                |
+
+## What this does and does not establish
+
+- It does not make Add fast at 16 concurrent writers: folding is still on the request path. The
+  measured cost of one fold (partition rewrite + bounded backfill) is what an accept pays when the
+  schedule lands on it.
+- `waitUntil` remains unverified for any other EdgeOne runtime; nothing here assumes it.
+- The 404 rows are still unexplained. `method` is what will explain them; the guess that they are
+  connectivity probes is a guess.
+
+## Alternatives considered
+
+- **Deploy the Pages Edge Function runtime to get `waitUntil`.** A different product and a different
+  deployment path; a migration is not a way to fix a fold budget in the same week as a scored run.
+- **Raise `FOLD_EVERY` so folds coincide less often.** That trades latency for a growing backlog,
+  which is the mechanism that hid accepted data in the last three rounds. The backstop stays folded.
+- **Stop folding on the request path and add a timer.** This runtime has no timer; a cron would be a
+  second deployment to operate, and the fold would still need to be bounded.
+
+## Consequences
+
+- One accept in `FOLD_TRIGGER` still pays the fold, but never for longer than the accept's own
+  budget: past it the reply is the same 200 with the log longer.
+- The fold's persisted "deferred" outcome is visible in `/admin/fold`'s counters, which is the
+  drain path a rollback needs.
+- `nmg_request` now carries `method`, `execution`: both are diagnostic-only and cost no round trip.
+- The two earlier records (`fold-runs-after-the-response`, `accept-without-cas`'s waitUntil
+  argument) are superseded on the platform claim; the mechanism they added (bounded, idempotent,
+  write-side folding) is what this record strengthens.

docs/decisions/implemented/2026-10-30-fold-budget-is-a-deadline.terms.yaml

@@ -0,0 +1,4 @@
+version: 2
+terms: []
+references:
+  - docs/decisions/implemented/2026-10-30-fold-budget-is-a-deadline.md

docs/decisions/implemented/2026-10-30-fold-budget-is-a-deadline.zh-CN.md

@@ -0,0 +1,49 @@
+# 折叠预算即截止时间；且该运行时不提供 waitUntil
+
+[English](2026-10-30-fold-budget-is-a-deadline.md) · [Terms](2026-10-30-fold-budget-is-a-deadline.terms.yaml)
+
+**Status:** implemented
+**Approved:** explicit
+
+## 问题
+
+两条事实，都来自生产日志行，而非读代码推断。
+
+**一：上一轮的前提对这个运行时是错的。** 稳定化部署之后，每次请求的常开日志行都报 `execution: "inline"`——适配器探测的 `context.waitUntil` 从未存在。在这类不提供该钩子的平台上是正确行为：Makers 的**Node 云函数** context 文档只列 `uuid / params / env / clientIp / server{region,requestId,geo}`。之前引用的 `waitUntil` 属于 **Pages 边缘函数**（`FetchEvent`）与 Pages 边缘 handler，和本项目部署的 Node 云函数不是同一个运行时。所以“折叠在响应之后跑”在生产从未生效；`2026-10-30-fold-runs-after-the-response` 中关于此的判断由本记录取代。15:00 那次 Add p50 1.02 s、尾部 6.5 s 是**内联**折叠在 1/8 排程下的形态，16:59 那两行同样写着 `execution: "inline"`。
+
+**二：折叠预算只管起点。** `if (dueBySchedule && Date.now() - startedAt < foldBudget)` 只决定折叠**要不要开始**；19.9 s 开始的折叠照样要把整个分区重写加 32 条回填跑完。这正是 14:09-14:15 的数字（Add 11-25 s、尾 100.7 s、5 个 120 s 被杀请求，栈全在 `foldPending`）的机制，前两轮没有覆盖，是因为没有任何东西在折叠开始之后约束它。
+
+同一批日志的第三个发现：两个 `/add`、`/search` 请求在**全冷 isolate** 上以 24-47 ms 返回 **404**（81 MB、`wasm_linear_mb: null`，从未碰 SQLite）。我们的 `/add`、`/search` 只接受 POST，所以那不是工作请求；请求行现在记录 `method`，让下次出现时是被识别而不是被猜。
+
+## 决策
+
+- **折叠接收截止时间并遵守它。** `foldPending(env, userId, { deadline })` 过了截止点不开始；在逐条应用之间检查，超时即停，把这一批剩余留在日志里。accept 传 `startedAt + foldBudget`；`/admin/fold` 传自己的 100 s。停下永远安全：条目已持久、折叠幂等、下一次排程的 accept 会重试。
+- **不依赖 `waitUntil`。** 分支保留给提供该钩子的运行时（测试提供），实际发布路径是有界的内联折叠——这也是本平台唯一支持的形态。
+- 请求行增加 `method`，仅用于诊断。
+
+## 证据
+
+| 改前                                                                         | 改后                                                                                     |
+| ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
+| 14:09-14:15：Add 11-25 s、尾 40-100 s、5 次 120 s 被杀、栈全在 `foldPending` | 超预算的折叠返回 `{ folded, remaining: null, deferred: "budget" }`，剩余条目等下一次排程 |
+| 15:00 p50 1.02 s / max 6.5 s：1/8 内联折叠，预算只管起点                     | 排程不变，但受调用方测量到的同一个时钟约束                                               |
+| 请求行没有 `method`；两个来源不明的 404                                      | 记录 `method`，404 要么可解释、要么能证明没有副作用                                      |
+
+## 这确立了什么、没有确立什么
+
+- 它没有让 16 并发写者下的 Add 变快：折叠仍在请求路径上，排程命中时 accept 仍要付一次折叠的成本。
+- 对其他 EdgeOne 运行时是否提供 `waitUntil` 未验证，本记录不假设任何一种。
+- 那两条 404 仍未解释；`method` 是让它被解释的手段，“连通性探测”目前只是猜测。
+
+## 考虑过的替代方案
+
+- **迁到 Pages 边缘函数运行时以取得 `waitUntil`**：这是另一个产品、另一条部署路径；迁移不是用来同周修一个折叠预算的手段。
+- **提高 `FOLD_EVERY` 让折叠更少发生**：那是拿积压换延迟，正是前三轮把已接收数据藏起来的机制；兜底仍是折叠，不是放弃。
+- **请求路径不再折叠、改用定时器**：本运行时没有定时器；cron 是第二套要运维的部署，而且折叠依然需要边界。
+
+## 后果
+
+- `FOLD_TRIGGER` 分之一的 accept 仍会付折叠，但绝不超过它自己的预算；超了照旧 200，日志更长。
+- 折叠“被推迟”的结果在 `/admin/fold` 的计数器里可见，而回滚需要的排空正是走那条路。
+- `nmg_request` 现在带 `method` 与 `execution`，两者都只用于诊断，不产生额外往返。
+- 早前两份记录（`fold-runs-after-the-response`、`accept-without-cas` 里的 waitUntil 论证）在平台事实上被取代；它们引入的机制（有界、幂等、写在写侧的折叠）由本记录加强。

onlion/backend/worker.mjs

@@ -743,7 +743,21 @@ async function inlineEntryVectors(env, messages) {
  *  Entries whose receipt already exists are skipped, which is what makes a fold
  *  that committed and then crashed before deleting its entries safe to repeat (I3).
  *  Their objects are removed only after the commit, never before. */
-async function foldPending(env, userId) {
+/** Folds one batch of waiting entries into the partition.
+ *
+ *  `deadline` is what makes the budget real. The caller's budget alone only decided whether the
+ *  fold *starts*; a fold that began at the last millisecond then ran the whole partition rewrite
+ *  and its backfill, which is how 2026-10-30 produced Adds at 40-100 s and five 120 s kills while
+ *  claiming a 20 s budget. Every expensive step now checks the clock first and stops, leaving the
+ *  rest of the log for the next scheduled attempt - the entry is already durable, so stopping is
+ *  always safe, and only the ordering of work changes.
+ *
+ *  This platform supplies no `waitUntil` to a Node cloud function (the EventContext documented
+ *  for Makers lists uuid/params/env/clientIp/server only), so folding after the response is not
+ *  available here; a bounded fold is what keeps the accept inside the invocation ceiling. */
+async function foldPending(env, userId, { deadline = null } = {}) {
+  const outOfTime = () => deadline !== null && Date.now() > deadline;
+  if (outOfTime()) return { folded: 0, remaining: null, deferred: "budget" };
   const client = embeddingClient(env);
   const key = keyFor(userId);
   const waiting = await listPending(env.MEMORY, key, { limit: FOLD_BATCH });
@@ -760,6 +774,7 @@ async function foldPending(env, userId) {
     // call: safe, but it looked like the drain had failed to make progress.
     return { folded: 0, remaining: released ?? pendingCount(manifest), released: true };
   }
+  if (outOfTime()) return { folded: 0, remaining: null, deferred: "budget" };
   const plan = foldPlan(waiting, { maxEntries: FOLD_BATCH, maxBytes: MAX_BODY_BYTES });
   let drop = [];
   let remaining = 0;
@@ -782,6 +797,12 @@ async function foldPending(env, userId) {
       }));
       // The entry may carry the vectors its accept computed. Handing them to this entry's client
       // means the fold applies storage and nothing else.
+      // Between entries, not inside one: half an entry cannot be applied. Stopping here leaves
+      // the rest of this batch in the log, where the next attempt folds it with its own budget.
+      if (outOfTime()) {
+        drop = names.size ? [...names] : [];
+        return { readOnly: drop.length === 0, response: { folded } };
+      }
       const seeded = seedFromEntry(env, entry);
       await addOneBatch(
         env,
@@ -2038,8 +2059,9 @@ async function foldRoute(request, env, url, path) {
   let folded = 0;
   let remaining = 0;
   let released = 0;
+  const drainDeadline = Date.now() + 100_000;
   for (let round = 0; round < 8; round += 1) {
-    const step = await foldPending(env, userId);
+    const step = await foldPending(env, userId, { deadline: drainDeadline });
     folded += step.folded;
     remaining = step.remaining;
     if (step.released) released += 1;
@@ -2076,19 +2098,21 @@ async function addRoute(request, env, path) {
     Number.parseInt(hash(accepted.response.request_id).slice(0, 6), 16) % foldEvery === 0;
   // The measured failure (2026-10-30 14:09-14:15 burst): folds riding inside the accept
   // pushed Adds to 11-25 s routinely and 40-100 s at the tail, and five requests died at
-  // 120 s - every one of them in foldPending. With an execution that can extend the
-  // invocation, the response goes first and the fold runs afterwards: the entry is already
-  // durable, the fold is idempotent (a crash before its entries are deleted repeats safely)
-  // and the next scheduled accept retries it, so a fold killed in flight costs nothing but
-  // one provider call, not a scored attempt. Inline folding stays the fallback for callers
-  // without waitUntil (tests, wrangler dev) and as the explicit escape hatch.
+  // 120 s - every one of them in foldPending. The budget above only decided whether a fold
+  // *started*, so a fold beginning at the last millisecond still ran a whole partition
+  // rewrite. It is now a real deadline handed to the fold, which checks it before each entry
+  // and leaves the remainder in the log - the entry is durable, so stopping costs nothing.
+  // The waitUntil branch is kept for callers that do provide one (tests, a future runtime):
+  // Makers' Node cloud function context lists uuid/params/env/clientIp/server and no such
+  // hook, and the always-on request line reports which path ran, as the execution field.
+  const foldDeadline = startedAt + foldBudget;
   const foldTask = dueBySchedule
-    ? foldPending(env, accepted.response.user_id).catch(() => {})
+    ? foldPending(env, accepted.response.user_id, { deadline: foldDeadline }).catch(() => {})
     : null;
   const execution = currentContext()?.execution;
   if (foldTask && execution?.waitUntil && env.NMG_FOLD_INLINE !== "1")
     execution.waitUntil(foldTask);
-  else if (foldTask && Date.now() - startedAt < foldBudget) await foldTask;
+  else await foldTask;
   return json(accepted.response);
 }
 async function handle(request, env) {
@@ -2205,6 +2229,7 @@ export default {
             at: new Date().toISOString(),
             trace_id: traceId,
             route,
+            method: request.method,
             status: responseStatus,
             elapsed_ms: Date.now() - startedAt,
             seq: context.seq,

onlion/scripts/test-add-log-mode.mjs

@@ -138,23 +138,32 @@ try {
     // The gated inline branch remains only as the test/dev fallback, so both shapes are pinned.
     assert.match(
       source,
-      /const dueBySchedule =[\s\S]{0,900}foldPending\(env, accepted\.response\.user_id\)/,
+      /const dueBySchedule =[\s\S]{0,2000}foldPending\(env, accepted\.response\.user_id, \{ deadline/,
       "the fold trigger must stay a schedule over request_id, not a counter",
     );
+    // This platform gives a Node cloud function no waitUntil (the Makers context lists
+    // uuid/params/env/clientIp/server), so the fold runs inline - and therefore the budget has
+    // to be a deadline the fold checks, not a gate on whether it starts. Both are pinned:
+    // the deadline is handed down, and the fold honours it before each entry it applies.
     assert.match(
       source,
-      /if \(foldTask && execution\?\.waitUntil && env\.NMG_FOLD_INLINE !== "1"\)[\s\S]{0,200}execution\.waitUntil\(foldTask\);/,
-      "when waitUntil exists the fold must run after the response, not inside it",
+      /const foldDeadline = startedAt \+ foldBudget;/,
+      "the accept must compute a real fold deadline, not just a start gate",
+    );
+    assert.match(
+      source,
+      /async function foldPending\(env, userId, \{ deadline = null \} = \{\}\) \{[\s\S]{0,1600}outOfTime\(\)\) return \{ folded: 0, remaining: null, deferred: "budget" \};/,
+      "the fold must refuse to start past its deadline",
     );
-    assert.doesNotMatch(
+    assert.match(
       source,
-      /await foldPending\(env, accepted\.response\.user_id\)/,
-      "the accept must never await a fold inline on the serving path",
+      /if \(outOfTime\(\)\) \{\s*drop = names\.size \? \[\.\.\.names\] : \[\];/,
+      "the fold must stop between entries, leaving the remainder in the log",
     );
     assert.match(
       source,
-      /else if \(foldTask && Date\.now\(\) - startedAt < foldBudget\) await foldTask;/,
-      "the inline fallback must still be budgeted; unbounded, a contended user folds past 120 s",
+      /if \(foldTask && execution\?\.waitUntil && env\.NMG_FOLD_INLINE !== "1"\)[\s\S]{0,200}execution\.waitUntil\(foldTask\);/,
+      "a runtime that does provide waitUntil must still be used",
     );
     // The root fix: an accept takes no compare-and-swap at all. Sixteen concurrent Adds on one
     // user used to serialise on the manifest and lose twelve times each, redoing a megabyte of