fix(runtime): preserve full UTF-8 context with measured 8 MiB RPC bounds

docs/reference/canonical-snapshot-pagination.md

@@ -23,8 +23,9 @@ continuation, so a large valid population remains readable without raising a
 transport limit or retrying a lost oversized response. The standalone wire-only
 `coordination.local_authority.todo_snapshot_page` endpoint remains compatible.
 Its pages, and capacity fallback pages, contain up to 4,096 Todo/lease records and
-1,792 KiB of UTF-8 JSON. The generic RPC response ceiling remains 2 MiB, leaving 256 KiB for its
-envelope. Normal collections can still complete in one page. A record
+1,792 KiB of UTF-8 JSON. That page budget still fits the original 2 MiB wire
+with 256 KiB for its envelope; it does not grow with the shared 8 MiB RPC ceiling
+described below. Normal collections can still complete in one page. A record
 and its acceptance guard travel together; records are never shortened to meet a
 budget. Archived dependency/decision records remain in the full collection.
 
@@ -118,3 +119,68 @@ This closes the bounded collection-read part of T3/L5. It does not establish D1
 permanent projection freshness, SQLite D2 durability/soak, D3 whole-Goal
 migration, or default-on qualification. See the
 [remaining PR sequence](../architecture/rfcs/shared-goal-authority-state-provider-v0.md).
+
+## Shared RPC byte budget
+
+Ordinary Python/TypeScript Effect requests and inline responses admit up to
+8 MiB, including the newline-delimited envelope. Python emits compact UTF-8
+JSON instead of ASCII-escaping every non-ASCII character. JSON values, escaped
+control characters and lone-surrogate compatibility remain intact. The server
+counts UTF-8 bytes and searches each new socket chunk for the delimiter, then
+parses the complete first frame once. Earlier chunks cannot contain that
+delimiter; repeatedly searching the accumulated string needlessly flattens it.
+
+This is an intentional increase from 2 MiB. A complete digest-bound source
+containing Chinese text and emoji occupied 1,000,387 UTF-8 parameter bytes but
+2,400,387 ASCII-escaped bytes: the old ordinary call refused before dispatch,
+while the explicitly allowed snapshot call wrote the inflated parameters.
+The UTF-8 call returns the same complete source, including its final constraint.
+It does not shorten Goal text or remove required reads to fit a budget.
+
+The capacity decision compared 2, 4 and 8 MiB on the same disposable Node 24
+worker with the UTF-8 encoder held fixed. Each trial read complete, hash-bound
+1/3/6 MiB sources, then four concurrent 6 MiB sources; a separate Python process
+sent small pings. Three uninstrumented trials per variant gave:
+
+| Ordinary cap and delimiter scan | 6 MiB warm roundtrip (ms) | Worker peak RSS (MiB) | Busy ping observed p95 (ms) |
+| --- | ---: | ---: | ---: |
+| 2 MiB, accumulated string; snapshot fallback | 68.7–79.5 | 196.8–218.0 | 20.8–26.8 |
+| 8 MiB, accumulated string; inline | 79.5–85.8 | 254.7–320.7 | 154.7–203.5 |
+| 8 MiB, new chunk only; inline | 57.7–60.0 | 214.6–222.4 | 28.0–66.8 |
+
+Raising the cap alone was rejected. V8 samples identified the repeated data
+callback work; changing only the delimiter scan removed that hotspot, and the
+uninstrumented comparison confirmed lower cost. The selected 8 MiB cap lets
+the complete 3/6 MiB sources stay inline. The 4 MiB candidate still sent the
+6 MiB source through a private file. Larger inline calls retain a concurrency
+cost: in a separate four-by-3 MiB workload, busy ping maxima were 72–101 ms
+with 8 MiB versus 17–19 ms with 2 MiB and snapshot fallback. These short local
+samples are observations, not a stable tail-latency SLO or long-horizon/model
+qualification. Roundtrip includes encoding, safe refusal/fallback when needed,
+transport and result validation; it excludes fixture preparation and startup.
+Peak RSS covers the worker's whole trial, including startup. The four large
+callers share one Python process; only the ping caller runs independently.
+
+The 64 MiB opt-in private-file capacity, method allowlist, same-UID ownership,
+permissions, digests, cleanup, deadlines and ambiguous-response recovery remain
+unchanged. Over-limit ordinary requests still refuse before dispatch; a possible
+commit followed by an oversized or lost response still requires receipt recovery.
+Pagination budgets and frozen experiment/promotion criteria remain independent.
+These transport changes add no capability, setting, execution or merge authority.
+Existing consumers that derive a limit from `MAX_REQUEST_BYTES` follow the same
+8 MiB budget: context-capture/native-input file and event readers, Todo summary
+and handoff batches, and task-graph/replan-history transport selection. Their
+source identity, file/privacy, digest and replay checks remain in force. Host
+loading limits, progressive presentation budgets and frozen experiment bounds
+are separate; increasing the shared wire does not change them.
+
+普通 Effect RPC 的请求和内联响应上限从 2 MiB 放宽到 8 MiB，按包含换行封包的
+实际 UTF-8 字节计量。保留完整正文、JSON 值及孤立代理项兼容，不删约束或必要读取。
+同时只扫描新增 socket 块中的换行，避免反复展开累积正文；单独提高上限的内存和
+延迟回退保留为失败对照。8 MiB 可让完整 3/6 MiB 来源走内联路径，但并发大请求仍
+可能增加小请求等待；以上为短时本机观测，不代表稳定尾延迟、真实模型或长程收益
+已验收。64 MiB 私有文件容量、方法和权限边界、摘要、清理、超时及不确定结果恢复
+不变；分页预算与已冻结实验／晋升条件独立，不追认历史失败。
+既有引用共享常量的上下文捕获／原生输入读取、Todo 批次与 task-graph/replan-history
+传输选择同步沿用 8 MiB；来源身份、文件隐私、摘要和重放检查保留。宿主加载和渐进
+展示预算各自独立，不随 RPC 上限放宽。

docs/reference/configuration-backup.md

@@ -489,7 +489,7 @@ SQLite gets a 64 MiB process-local [hard heap limit](https://www.sqlite.org/prag
 this is why that verifier is not dispatched in the shared Effect daemon. Views
 and virtual tables cannot masquerade as authority tables. WAL-mode copies and
 unsupported/service-owned providers remain unknown and need their existing
-owner; PostgreSQL is never contacted. The normal 2 MiB RPC wire is unchanged;
+owner; PostgreSQL is never contacted. The shared [8 MiB ordinary RPC wire](canonical-snapshot-pagination.md#shared-rpc-byte-budget) remains bounded;
 only the two complete-manifest checks opt into the existing 64 MiB private-file
 transport. These unchanged bounds are approved for local POSIX File/SQLite
 inert inspection by the [recorded maintainer decision](../architecture/rfcs/complete-state-recovery-v0.md#m1-capacity-default-decision).

docs/reference/protocols/goal-vision-replan-contract-v0.md

@@ -1007,7 +1007,7 @@ configured, the current agent vision, and the local source binding. A replan
 obligation covers the full Todo frontier. Archived dependencies remain inputs.
 Large local bases use digest-checked private files across the Python/TypeScript
 runtime boundary, including the response; the CLI still returns the complete
-basis. The 2 MiB default RPC guard remains for other effects. File size is
+basis. Other effects retain the shared [8 MiB RPC guard](../canonical-snapshot-pagination.md#shared-rpc-byte-budget). File size is
 bounded and an unverifiable response after a possible commit is ambiguous,
 so the caller reads the exact receipt before retrying any mutation. Neither
 transport nor a future paged presentation may silently omit a basis component.

docs/reference/protocols/loopx-turn-v0.md

@@ -522,7 +522,7 @@ to preserve integral floats, signed zero and unsafe integers before atomic
 publication. Replay and immutable-plan comparison distinguish these source
 numbers; refusals retain the original file. Ordinary objects without metadata
 retain their encoding and operation identity; historical receipts are not
-rewritten. This remains one RPC under the existing 2 MiB request limit and avoids
+rewritten. This remains one RPC under the shared [8 MiB request limit](../canonical-snapshot-pagination.md#shared-rpc-byte-budget) and avoids
 double escaping evidence text. Metadata still costs bytes for number-dense
 sources. Source fidelity and transport checks do not establish latency or
 long-horizon benefit.
@@ -531,7 +531,7 @@ Python 来源的 journal 快照仍经同一 TypeScript writer 传递 JSON 对象
 context delivery 的稀疏数值 IO 元数据与校验器，在原子发布前保留整值浮点、带
 符号零和非安全整数。重放与不可变 plan 比较区分这些来源数值；拒绝时保留原
 文件。无元数据的普通对象保留已有编码与操作身份，不改写历史回执。仍为一次
-RPC，沿用 2 MiB 请求上限，避免证据正文的双重转义；数值密集来源仍有元数据
+RPC，沿用共享的 8 MiB 请求上限，避免证据正文的双重转义；数值密集来源仍有元数据
 字节成本。来源保真与传输检查不代表延迟或长程收益已得到验证。
 
 If executing `run-once` raises unexpectedly, its error response retains the

docs/reference/research-checkpoint.md

@@ -45,7 +45,7 @@ files; arbitrary caller-declared artifacts remain private and are not privacy
 certified. A new destination is required; capture never overwrites a point.
 
 Artifact operations use the existing private local snapshot transport for full
-inventories and retained metadata above the ordinary 2 MiB Effect wire limit.
+inventories and retained metadata above the ordinary [8 MiB Effect wire limit](canonical-snapshot-pagination.md#shared-rpc-byte-budget).
 The existing 64 MiB local snapshot bound and original operation deadlines still
 apply. No witness is truncated, no capture is enabled implicitly, and a lost
 capture response must be reconciled under its original identity before another

docs/reference/reviewed-coordination-promotion.md

@@ -228,13 +228,16 @@ an execution fails; do not infer that a failure left legacy writers usable.
 
 ## Bounded capture proof transport
 
-A long Goal can exceed the existing 2 MiB RPC response budget before promotion:
+At the original 2 MiB RPC response budget, a long Goal could exceed it before promotion:
 sequence recovery used to return a full head and full projections for retained
 transactions. The `outbox_read` proof read model now keeps full lineage validation
 inside TypeScript, while returning progress, receipts, projection digests and
 partition markers. Sequence allocation requests no transaction rows; drain uses
 the compact rows. Existing full diagnostic reads retain their default contract.
-No transport limit, stored population or transaction validation is weakened.
+That proof-read change did not raise a transport limit, reduce the stored population
+or weaken transaction validation. The ordinary wire now uses the independently
+measured [shared 8 MiB budget](canonical-snapshot-pagination.md#shared-rpc-byte-budget);
+the original refusal remains historical evidence.
 A pending outbox still blocks promotion; use the existing bounded
 `authority-shadow drain --goal-id example-goal --budget-seconds 60` operation
 and inspect its result before retrying preview.

docs/reference/reviewed-coordination-promotion.zh-CN.md

@@ -173,10 +173,12 @@ provider revision、receipt 内容和初始 projection 必须全部一致。
 ## 大 Goal 的捕获证明
 
 序号恢复以前会返回完整 head 和历史事务的完整 projection，大 Goal 可能因此超过
-现有 2 MiB RPC 响应上限，甚至不能产生晋升所需的 shadow 变更。
+当时的 2 MiB RPC 响应上限，甚至不能产生晋升所需的 shadow 变更。
 现在 `outbox_read` 的 proof read model 在 TS 内完整校验事务链，只传回进度、receipt、
 projection digest 和 partition marker；分配序号不返回事务行，drain 使用紧凑行。
-原有完整诊断读取保持默认合同，不提高传输上限、不减少持久数据，也不省略链校验。
+原有完整诊断读取保持默认合同；该 proof-read 变更不提高传输上限、不减少持久数据，
+也不省略链校验。普通传输现沿用单独测量的[共享 8 MiB 预算](canonical-snapshot-pagination.md#shared-rpc-byte-budget)，
+原超限仍保留为历史失败。
 若 outbox 尚未结清，晋升仍会拒绝。先执行并检查既有的有界 drain：
 `authority-shadow drain --goal-id example-goal --budget-seconds 60`。
 

docs/reference/trajectory-assessment.md

@@ -789,7 +789,7 @@ counter observations are differenced across Turns; summing the cumulative
 part of input, and reasoning is part of output, not another additive cost.
 
 Complete counter histories use the existing private local snapshot transport
-when they exceed the ordinary 2 MiB RPC wire. Only this explicitly allowed
+when they exceed the ordinary [8 MiB RPC wire](canonical-snapshot-pagination.md#shared-rpc-byte-budget). Only this explicitly allowed
 method opts in; reduced snapshots retain the 64 MiB ceiling and digest checks.
 Oversized captures fail explicitly rather than dropping earlier Turns.
 

loopx/capabilities/performance_diagnosis/README.md

@@ -82,7 +82,7 @@ Input files are limited to 16 MiB; shorten captures beyond that limit. Repeated
 profile names and hotspot labels share a 1 Mi-character display-text budget;
 oversized summaries are rejected explicitly, never silently truncated. Larger
 recordings reuse Effect's private local snapshot transport without raising the
-ordinary 2 MiB message limit or returning the raw profile in the summary.
+ordinary [8 MiB message limit](../../../docs/reference/canonical-snapshot-pagination.md#shared-rpc-byte-budget) or returning the raw profile in the summary.
 Unknown source locations exported as `file: null` or `line: null` are omitted;
 incorrect non-null types remain errors. Repeated V8 leaves are aggregated before
 ancestor traversal, preserving sample counts and self/inclusive weights. One

loopx/capabilities/performance_diagnosis/README.zh-CN.md

@@ -50,7 +50,7 @@ Memray/V8 heap 用工具自己的 allocation reporter。其余工具是后续诊
 进程执行时长。
 输入限 16 MiB，超出时缩短采集；重复展示的 profile 名称与热点标签共用 1 Mi 字符
 上限，超出时明确拒绝，不静默截断。较大的录制复用 Effect 的私有本地快照传输，
-不会提高普通命令的 2 MiB 传输上限，也不会把原始 profile 返回到展示结果。
+不会提高普通命令的 [8 MiB 传输上限](../../../docs/reference/canonical-snapshot-pagination.md#shared-rpc-byte-budget)，也不会把原始 profile 返回到展示结果。
 导出器用 `file: null` 或 `line: null` 表示未知位置时，读取器按缺省处理；非空
 错误类型仍拒绝。V8 重复 leaf 先聚合再遍历祖先，保留原样本数及 self/inclusive
 权重。整份请求的 frame 分配、sample/event 访问与栈展开共用 2,000,000 工作单位

loopx/control_plane/effect_runtime.py

@@ -40,8 +40,8 @@
 EFFECT_RUNTIME_STARTUP_ERROR_SCHEMA_VERSION = (
     "loopx_effect_runtime_startup_error_v0"
 )
-MAX_RESPONSE_BYTES = 2 * 1024 * 1024
-MAX_REQUEST_BYTES = 2 * 1024 * 1024
+MAX_RESPONSE_BYTES = 8 * 1024 * 1024
+MAX_REQUEST_BYTES = 8 * 1024 * 1024
 MAX_LOCAL_SNAPSHOT_BYTES = 64 * 1024 * 1024
 LOCAL_SNAPSHOT_METHODS = frozenset({
     "todo.read",
@@ -945,6 +945,13 @@ def _stop_effect_runtime(
     }
 
 
+def _encode_rpc_json(value: Mapping[str, Any]) -> bytes:
+    # Count and transmit UTF-8 bytes; lone surrogates retain valid JSON escapes.
+    return json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode(
+        "utf-8", errors="backslashreplace",
+    )
+
+
 def _request_with_info(
     info: Mapping[str, Any],
     *,
@@ -974,9 +981,9 @@ def _request_with_info(
             descriptor = os.open(response_sink, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
             os.close(descriptor)
             request["response_sink"] = str(response_sink)
-            encoded = (json.dumps(request, separators=(",", ":")) + "\n").encode()
+            encoded = _encode_rpc_json(request) + b"\n"
             if len(encoded) > MAX_REQUEST_BYTES:
-                params_bytes = json.dumps(dict(params), separators=(",", ":")).encode()
+                params_bytes = _encode_rpc_json(dict(params))
                 if len(params_bytes) > MAX_LOCAL_SNAPSHOT_BYTES:
                     raise EffectRuntimeRejected(
                         "TypeScript Effect runtime local snapshot is oversized",
@@ -992,9 +999,9 @@ def _request_with_info(
                     "path": str(params_path), "byte_count": len(params_bytes),
                     "sha256": hashlib.sha256(params_bytes).hexdigest(),
                 }
-                encoded = (json.dumps(request, separators=(",", ":")) + "\n").encode()
+                encoded = _encode_rpc_json(request) + b"\n"
         else:
-            encoded = (json.dumps(request, separators=(",", ":")) + "\n").encode()
+            encoded = _encode_rpc_json(request) + b"\n"
         if len(encoded) > MAX_REQUEST_BYTES:
             raise EffectRuntimeRejected(
                 "TypeScript Effect runtime request is oversized",

loopx/control_plane/effect_runtime_server.ts

@@ -24,9 +24,9 @@ const REQUEST_SCHEMA = "loopx_effect_runtime_request_v0";
 const RESPONSE_SCHEMA = "loopx_effect_runtime_response_v1";
 const INFO_SCHEMA = "loopx_effect_runtime_info_v0";
 const STARTUP_ERROR_SCHEMA = "loopx_effect_runtime_startup_error_v0";
-const MAX_REQUEST_BYTES = 2 * 1024 * 1024;
-const MAX_INLINE_RESPONSE_BYTES = 2 * 1024 * 1024;
-// Explicit opt-in: ordinary effects retain the 2 MiB request/response wire.
+const MAX_REQUEST_BYTES = 8 * 1024 * 1024;
+const MAX_INLINE_RESPONSE_BYTES = 8 * 1024 * 1024;
+// Explicit opt-in: ordinary effects retain a bounded request/response wire.
 const LOCAL_SNAPSHOT_METHODS = new Set([
   "todo.read",
   "todo.quota_planning.project",
@@ -186,14 +186,16 @@ const server = createServer((socket) => {
         request_id: "unknown",
         ok: false,
         error: effectRuntimeErrorPayload(new EffectRuntimeRequestError(
-          "Effect runtime request exceeds the 2 MiB limit",
+          "Effect runtime request exceeds the 8 MiB limit",
           "request_too_large",
         )),
       })}\n`);
       return;
     }
     raw += chunk;
-    if (!raw.includes("\n")) return;
+    // Earlier chunks had no delimiter. Scanning the accumulated string on each
+    // chunk repeatedly flattens it; only the newly received chunk can finish it.
+    if (!chunk.includes("\n")) return;
     socket.pause();
     pendingRequests += 1;
     clearTimeout(idleTimer);

tests/capabilities/test_trajectory_assessment_store.py

@@ -1156,7 +1156,7 @@ def test_large_usage_real_cli_register_readback_and_replay(pack, monkeypatch):
     )
     assert (
         len(json.dumps(params, separators=(",", ":")).encode())
-        > effect_runtime.MAX_REQUEST_BYTES
+        > 2 * 1024 * 1024  # Keep the original complete 4,500-Turn counter history.
     )
     registry = sources.parent / "registry.json"
     registry.write_text('{"projects":{}}')
@@ -1220,7 +1220,9 @@ def cli(*args):
     assert "solver_usage" not in inspect_pack(**scope, pack_id=old["pack_id"])
     assert view["job"] is None and view["result"] is None
 
-    # The method opts into a bounded channel, not a larger ordinary RPC wire.
+    # Keep the original 2 MiB refusal and reduced private-channel negative.
+    # The complete counter history above qualifies the current default wire.
+    monkeypatch.setattr(effect_runtime, "MAX_REQUEST_BYTES", 2 * 1024 * 1024)
     with pytest.raises(
         effect_runtime.EffectRuntimeRejected, match="request is oversized"
     ):

tests/control_plane/test_canonical_snapshot_integration.py

@@ -63,7 +63,7 @@ def wide_goal(tmp_path, monkeypatch, request):
             "status": "open",
             "done": False,
             "text": f"Retained work {index}",
-            "note": "完整🙂" * 1600,
+            "note": "完整🙂" * 6000,
             "archive_state": "active",
             "source_section": "Agent Todo",
             "index": index + 1,