Skip to content

test: make wall-clock-dependent molecule/harness/process tests deterministic (#220) - #226

Merged
DanielNetzer merged 1 commit into
mainfrom
forgecell/issue-220-deterministic-tests
Oct 25, 2026
Merged

DanielNetzer merged 1 commit into
mainfrom
forgecell/issue-220-deterministic-tests

Conversation

@DanielNetzer

Copy link
Copy Markdown
Owner

Closes #220.

This is a test-only change across molecule, harness, process and cli. No production file changes, and every shipped default is untouched.

  • Readiness markers instead of wall-clock waits: a new cancelWhenStarted helper cancels a run only after the stub has written its "started" marker. Interruption tests no longer race a 10 s watchdog, and a stub that never starts fails explicitly.
  • Marker-driven expiry in process: expireOnMarker and markedShell let tests that need output and then a timeout expire only after the output provably exists. Silent-stall tests still wait out a real stallTimeout, because there the deadline is the behavior under test.
  • Success backstops: tests that only need a run to finish use a named 2-minute successBackstop instead of a 10–30 s budget. That replaces the 10 s budget in TestNativeBindingPassesAllowlistToClaudeArgv and the 30 s budgets in analysis_correction_test.go. The shared run fixture's check timeout goes from 5 s to 120 s.
  • TestHarnessTimeoutPlanValueIsHonouredAndRecorded: the control run now uses a stub that never finishes, so it still proves the Formula timeout fires. The plan-extended run gets minutes of headroom over its 11 s harness, and the test asserts that headroom.

Built by Forgecell Molecule mol-220-05bba754ffa48566e90b815a (plan revision 2 added the three extra flaky files the coder found). On the candidate tree, the load acceptance go test -count=5 -p 8 ./internal/molecule/... ./internal/harness/... ./internal/process/... passed, along with the 10× repeat of the three named tests, the cli/install packages and the shipped-default assertions.

Hand-delivered: the Lab's full-suite baseline failed only on Go's 10-minute timeout in the unrelated cli and evaluation packages, which #222/#224 plus the Formula refresh address. Rebased onto main (1f57972), an uncached clean-shell go test -count=1 -timeout 25m ./... passes, and gofmt and vet are clean.

🤖 Generated with Claude Code

…inistic (#220)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DanielNetzer
DanielNetzer merged commit 4beed03 into main Oct 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v1 blocker: make wall-clock-dependent molecule/harness tests deterministic (they keep failing the Lab baseline)

1 participant