Skip to content

fix: qualify observed Codex replay profiles and native validation - #7785

Open
loopx-agent wants to merge 4 commits into
mainfrom
codex/native-replay-profile-20261101
Open

loopx-agent wants to merge 4 commits into
mainfrom
codex/native-replay-profile-20261101

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

Current native qualification had three reproducible failures: the full-file oracle opened a retired duplicate after #7775; replay of exact observed Codex writer metadata 0.163.0-alpha.11.2 / 0.166.0-alpha.5.1 lacked a resume witness; and test runtime retirement could remove a directory during locator traversal. This proposal fixes those existing caller paths.

  • Before → after on the same 16 actual native cases with Codex 0.166.0-alpha.5.1: main bc8658523 failed all 16 on the obsolete file oracle; the fixture-only commit 89d34dca2 passed 10 and failed 6 resume cases; final 4b0037065 passed all 16. These earlier failures remain evidence.
  • Base: main, selected at bc8658523968d2ff5c59d56afeb920fbe95f71ef. Subsequent main changes through 6e33f2c67 touch three unrelated test files; no overlapping source changes.
  • Related: C1: converge complete Turn decisions and inspection callers #7430 remains a separate unfinished C1 proposal. This repair does not qualify its default-output cost or complete its acceptance.

Author Declaration

  • Written by: model_agent; OpenAI GPT-6 (Codex).
  • Specification: no separate written specification; the reproduced defects and acceptance below are the basis. The existing native replay/descriptor owners at base bc8658523 supply the compatibility boundary.
Criterion Disposition Owning path Decisive evidence
Recognize only the two exact observed writer versions implemented codex_context_history._snapshot Positive resume controls and actual native resume, replaced/unqualified history, and two-compaction recovery
Keep replay source, metadata, ownership, digest, read-completion and loss checks implemented Existing replay owner, unchanged record/event sets Full loss/refusal matrix runs for each added version; unknown next versions remain unqualified
Observe the advertised full/delta file and its digest implemented test_codex_context_capture.py Same actual 16-case progression; no recreated delivery.json
Tolerate disappearing disposable branches without hiding unexpected traversal IO failures implemented managed_effect_runtime_fixture.py Deterministic failing-before race, injected IO failure, plus ownership/symlink/PID and nested runtime controls

Self-review keeps this as a provider compatibility edit and two directly demonstrated test repairs. The small traversal error callback owns the known-retirement versus unexpected-error boundary; no general abstraction or new decision owner is needed. The test-only full-wire normalization retains independent bytes/digest/target and model-input oracles. New record/event forms, future-version ranges and protocol relaxations are outside this proposal.

Scope And Continuation

Complete within this compatibility scope. Native context delivery remains opt-in and the existing CLI/native-host journey remains the caller; no new capability, configuration owner, frontend control or layout is introduced. No first-screen visual evidence is applicable. Packaged native/CLI readback is distinguished from installed App use and model comprehension.

The parent C1 work remains open in #7430: current-main integration, default plan-carrier duplication/output budget, complete reader exit and outcome/full-cost comparison still require their own evidence. This independently reversible repair does not depend on completing that larger wire change.

Validation

  • Tested revision: 4b0037065f25f7f0f2eca92767525e6e382c8993; profile controls ran on identical blobs before the final commit.
  • Run state: finished
  • Input classes: synthetic, public_fixture. Actual Codex runs use a deterministic loopback provider; no external model request or benchmark job.
Check kind Result Evidence / limitation
regression_parity passed Same native 16-case sequence above: 16 obsolete-oracle failures → 10 passed/6 resume failures → 16 passed (271.58 s). Binary version stable before/after final run. These are correctness observations, not a latency comparison.
unit passed Cleanup and replay-profile controls: 109 passed (49.41 s), including malformed/loss/rollback/unknown/output/identity/permission cases.
unit passed Capture/lifecycle modules: 163 passed, 46 opt-in native cases skipped (34.34 s); the selected actual native matrix is reported separately.
real_entrypoint passed Actual exec, app-server and CodeMode; process resume, worker history replacement/unqualified profile, two-compaction working brief recovery.
real_entrypoint passed 10 passed (133.42 s) through an isolated final wheel: normal/resume plus missing/killed-hook and public hook-CLI journal recovery. Imports verified from installed site-packages; binary version stable before/after. No live install.
static passed Ruff on all five changed Python files; configured mypy (17 files); control-plane TypeScript typecheck; bilingual diff and whitespace checks.
manual passed Exact five-file scope 1b20a5b73f1172b3efcd19c97be2fa5c1da7a9bbbc60af7839c492477deee2d5; receipt cqr_1b20a5b73f1172b3efcd valid, zero blockers; safe-fix allowed / unused (0 passes). The optional inherited failure below remains a failed observation with matched attribution.
integration passed Standard premerge gate: six direct and ten selected checks executed; no blocking failure. The full-tree catalog advisory below is disclosed separately.

| static | failed | Optional full-tree maintainability advisory: unchanged oversized host_process_transport.run_host_process. Independent clean immutable base 6e33f2c67 and head 4b0037065 runs both exit 1 with identical full diagnostics, fixture/environment digests and zero magnitude regression. Exact-diff maintainability passes; required checks are not waived. |

Coverage limits: native transport/input-boundary behavior is exercised with the real binary and owning backend; inference is deterministic. No claim of autonomous adoption, retained model comprehension, solver benefit, installed App behavior or full C1 delivery. No PostgreSQL store, runtime authority, quota, scheduler, permissions or score semantics are changed. CI was neither queried nor manually triggered. Merge and live installation remain separate maintainer actions.

All commits are DCO-signed. No raw native histories, temporary evidence, generated bundle, credentials or local operating context is included.

Review Repair At 37facf0da9d9165e6683ae6a61dddb82120a1aa0

The branch is rebased onto da4510af61277c0d0786fa5aa040ccb01538b1cb, so the existing enclosing-tool-failure protection from main is now part of the tested head. A real quota context-hook CLI regression covers both newly qualified writer versions: an outer isError: true wrapper containing otherwise valid reader pages retires the candidate, does not create native_resume, and SessionStart recovery keeps the candidate absent. Normal qualified replay and unknown-version fallback remain covered.

Validation at the exact head: 278 passed, 46 explicit native-binary skips across native resume, lifecycle, capture and managed-runtime fixture modules; the focused failure/normal/version matrix passed 14/14; Ruff, diff check, semantic inventory advisory and public-boundary literal scan passed. The first broad run used an unintended system Python without _sqlite3 or nested pytest and is retained as environment-failure evidence; the declared Python 3.12 plus test extra run is the passing result. No live install, external model call or benchmark job was performed.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; model=gpt-6.1-sol; provider=OpenAI; runtime_reported; reasoning_effort=max

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

发现一项阻塞:[P2] 纳入工具失败保护后再资格化新版 native replay。在新增资格的 0.166.0-alpha.5.1 下,PostToolUse 的 tool_response 外层为 isError: true、内层含完整成功 reader JSON 时,真实 hook CLI 仍建立 candidate.native_resume,后续 SessionStart(resume) 保留 candidate。相同 fixture 的基线不为此 writer 建立见证;当前 head 却将失败工具结果纳入了新增资格。现有合同要求已知读取失败不能建立 baseline。已读取主干 62390312506d0087b4bd9433e59ef24c01ea6235 对 _context_pages 的外层失败保护,但它尚未包含在本 PR 的被测 head。请以包含该保护的新 head 重跑真实 hook CLI 的失败 wrapper 反例;不需要重复实现已落地的主干修复。

动机 / Motivation

已显式启用 native context delivery,并用新版 Codex 恢复会话的用户。旧版本不能为两个新 writer 建立 native replay 见证,需要走恢复路径;当前新增精确版本后,正常恢复已能继续,但工具结果外层已报告失败时,内层成功 JSON 仍可能被误当成可复用读入。当前正常 native 矩阵 16 项通过,独立 hook CLI 的关闭与损失控制也通过;外层工具失败反例仍失败,因此这个 head 尚不能完成新版 replay 资格验收。范围是已有 provider 的精确版本兼容和验证修复;不宣称完成 C1、提升默认配置、证明安装后的 App 使用、模型理解或长期收益。

改动思路 / Approach

writer 兼容仍归现有 Python IO 观察器,baseline 复用决定仍归现有 typed lifecycle owner,测试应读取实际公布的文件。本 PR 保留精确版本白名单与原有 IO/权限边界;完成资格前需要把已有的工具失败判定纳入被测 head,不需要新建 replay owner。将两个实际观察到的精确 writer 字符串加入已有 IO 兼容分支,比版本范围或新的状态 owner 更合适。读取 advertised 文件并核对 digest,也修复了测试仍打开已退役 delivery.json 的旧 oracle;测试专用目录遍历则需要同时保留已消失分支的容错与意外 IO 错误的可见性。

正常路径的价值已见于真实 exec、app-server、CodeMode 和合成 worker 入口:读取原始完整文件、接续 delta、进程 resume 和 compaction 后恢复。问题在于新增资格打开了依赖 _context_pages 的已知错误输入路径。正常矩阵通过不能替代“外层失败必须否决内层成功 JSON”的负例;关闭功能的隔离通过也不能解决开启后的资格缺口。

具体改动 / Changes

完整审阅 5 个文件,+82/-14,生产变化仅 +3/-1,其余是 capture/resume 与测试 runtime census。当前 head 4b0037065f25f7f0f2eca92767525e6e382c8993;本次按实际 merge base bc8658523968d2ff5c59d56afeb920fbe95f71ef 比较,未将 API 的后续目标状态 6e33f2c67e78f0548e9a3755a2f066fcf02bff5f 冒充相同基线。

已接受合同是 native context delivery 指南,固定 revision bc8658523968d2ff5c59d56afeb920fbe95f71ef。NATIVE-OFF:implemented,关闭时原 journal 字节不变;NATIVE-READ-LOSS:not_met,已知工具失败仍能为新增 writer 建立 resume 见证;NATIVE-CAPTURE:implemented,实际公布文件的字节、digest、target 及完整重建已检查。C1 的路线图边界仍要求后续消费者、结果与完整成本资格,本修复不承担整个父项完成。

关键代码讲解

  1. _snapshot 新增 0.163.0-alpha.11.2 和 0.166.0-alpha.5.1 两个精确 writer。物理绝对路径、文件归属/权限、metadata、记录边界和变化检测保留,未知后续版本不因前缀相近而自动获得资格。

  2. codex_context_hook 读取页、交给既有 typed lifecycle 更新,再由 IO 观察器附加 native replay 见证。新版本通过 metadata 分支后,正是沿这条真实路径受到外层失败漏判影响;版本字符串从来不是工作或权限授权。

  3. _context_pages 在本 head 的 content 分支只提取内层 JSON,没有否决外层 isError。独立 probe 使用真实 hook CLI,传入外层失败和完整成功页,观察 candidate.native_resume 与 resume 后 candidate,复现了上面的阻塞。已读取主干的对应保护,没有声称已在本 PR head 上验证修复。

  4. capture_disposable_effect_runtimes 改用局部 os.walk 错误回调;仍排除借用目录、符号根和越界 locator。独立文件系统控制在本机对“目录消失”于 base/head 都能继续,因此作者的该项基线失败没有在本机复现;可确认的是 EIO 从旧路径静默吞掉变为当前抛出。既有真实 runtime 清理/身份控制也已执行。

同输入观察 base 当前 head
真实 Codex 16-case capture/resume 矩阵 16 FAIL,保留旧 oracle/候选见证等原始失败 16 PASS,0 SKIP
0.166.0-alpha.5.1,正常 reader 无 native 见证,resume 后 candidate 移除 建立见证并保留,符合兼容修复目的
同版本,外层 isError: true + 内层完整成功 JSON 无 native 见证,candidate 移除 仍建立见证并保留:FAIL
原有 0.162.1 的相同错误 wrapper 也能保留 仍能保留,作为既有错误的控制;新增版本扩大了资格范围
14 组 profile/loss、56 次实际 hook CLI 与对应旧资格状态比较 常规控制 PASS;关闭返回值与 journal 字节保留
replay 与清理控制、静态检查 — 109 PASS;Ruff、配置 mypy 17 文件、编译和语义检查 PASS
标准 premerge — 6 项直接与 9 项目录检查 PASS;1 项既有维护性 advisory 仍 FAIL

对主干的风险 / Risks

P2 的触发形状为 PostToolUse.tool_response = {"isError":true,"content":[{"type":"text","text":"<完整成功页 JSON>"}]}。在已读合同中,已知读取失败不能建立 baseline。当前新 writer 却能在这个实际 CLI 输入下得到 native_resume,后续 resume 保持原 epoch/candidate。不是将任意旧缺陷要求本 PR 重修:同一新增 writer 在基线没有该见证,本次资格扩展使它进入了这个已知失败路径。最小修复是纳入已有主干保护,并用同一失败 wrapper 加正常控制重验新 head;无需重做已落地修复或扩大到完整 C1。

正常 native 检查使用真实 Codex 0.166.0-alpha.5.1、真实 owning backend 和隔离 runtime;响应由固定 loopback 服务生成,没有外部模型推理、真实 benchmark 任务或全局安装。独立 profile 控制还区分了 native 见证与仍合法的 saved-file delta:文件缓存可重建,不等于模型保留。真实 binary 验证使用 Codex 0.166.0-alpha.5.1 和固定的本地响应;0.163.0-alpha.11.2 使用合成 writer 控制。未验证安装后的 App、真实模型保留/理解、净费用收益或完整 C1。

早期独立脚本曾因非物理临时路径、目录遍历注入器未转交 FD、错误假设基线也抛 EIO,以及 hook probe 漏带显式隔离路由而失败。记录均保留;修正的是 fixture/adapter,当前真实外层失败反例没有被改成通过。目录消失的作者历史原因继续保持未独立复现,不能从后来通过推断已解释。

维护性 ratchet 仍 FAIL:基线/当前相同命令的完整失败 SHA256 均为 9d88719f4a1548b47726f28ad226ff76e2ef1844c616d83f2ce46170a14838fe,不变 oracle/预算/报错模块和环境证据已核对,未降低门限。它与本次新发现分别记录:前者是已有债务,后者是必需负例的真实失败。CQR 已记录 fail 并保留阻塞,没有改写成通过。本轮未查询、轮询或等待 CI。

语义与 CI 对齐

默认关闭路径的实际配对控制保持隔离,没有新增 prompt、必填字段、设置或调度义务。writer 版本是 provider 的外部兼容输入,复用原有语义,不是新的角色生命周期或权限协议。当前违反的是既有读取失败监督合同;差异级词汇 advisory 和其他通过检查不能覆盖这个缺口,也不能把必须拒绝的失败称为建议。

我的整体评价 / Verdict

not_yet_proven 指当前新版 replay 的完整资格,不否定正常恢复和测试 oracle 修复的实际价值。长程连续性与用户体验当前均为 not_yet_proven:正常路径能继续,但错误工具结果仍可能被当成可复用见证;最小补齐是包含已有保护的新 head 及同一失败/正常 CLI 控制。普通使用没有新增确认轮次,问题是机器对失败状态的判断,不是要求用户多做配置或确认。

已检查 native hook/provider、原有 context-config 路由和测试修复;没有渲染呈现或布局改动,视觉评审不适用。真实 binary 验证使用 Codex 0.166.0-alpha.5.1 和固定的本地响应;0.163.0-alpha.11.2 使用合成 writer 控制。未验证安装后的 App、真实模型保留/理解、净费用收益或完整 C1。writer 兼容仍归现有 Python IO 观察器,baseline 复用决定仍归现有 typed lifecycle owner,测试应读取实际公布的文件。本 PR 保留精确版本白名单与原有 IO/权限边界;完成资格前需要把已有的工具失败判定纳入被测 head,不需要新建 replay owner。范围和代码量相称,局部兼容与测试修复可以保留;需要修复的是开启后的资格负例。C1 的默认输出成本、完整 reader 退出、真实采用与结果/完整费用继续是父项的独立未验证边界,不能由此 PR 的测试数量代替。

English verdict: REQUEST_CHANGES at 4b0037065f25f7f0f2eca92767525e6e382c8993. The newly qualified writer retains native replay after an enclosing failed tool result; include the existing decoder protection and revalidate the exact head.

loopx-agent and others added 4 commits November 1, 2026 20:07
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: huangruiteng <huangrt01@163.com>
@huangruiteng
huangruiteng force-pushed the codex/native-replay-profile-20261101 branch from 4b00370 to 37facf0 Compare November 1, 2026 12:13
@huangruiteng

Copy link
Copy Markdown
Collaborator

Resolved the P2 at exact head 37facf0da9d9165e6683ae6a61dddb82120a1aa0.

The branch now includes main's enclosing isError/failed-tool protection. I added real hook CLI negative controls for both newly qualified writers (0.163.0-alpha.11.2 and 0.166.0-alpha.5.1): valid reader JSON nested inside a failed tool result cannot establish a candidate or native_resume; resume returns recovery and keeps the candidate absent. Normal qualified writers and unknown next versions remain covered.

Validation: 278 passed, 46 explicit native-binary skips; focused matrix 14 passed; Ruff, diff check, semantic inventory advisory and public-boundary scan passed. The earlier broad attempt used an unintended Python missing _sqlite3/nested pytest; rerunning with the repository's Python 3.12 and test extra passed.

huangruiteng added a commit that referenced this pull request Nov 1, 2026
Signed-off-by: huangruiteng <huangrt01@163.com>
huangruiteng added a commit that referenced this pull request Nov 1, 2026
Signed-off-by: huangruiteng <huangrt01@163.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants