Skip to content

test(mothership): drive a draft run past a wait slice through the real controller - #9475

Merged
waleedlatif1 merged 2 commits into
stagingfrom
test/live-sliced-draft-run
Oct 30, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
test/live-sliced-draft-run

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a live acceptance case to saved-run-read.live.test.ts: a long draft test run goes through the real controller against the worker's sliced synchronous workflows run. It builds on #9464 (workflows run --wait-timeout and the workflow-run-wait-timeout capability) and #9469 (a still-running slice settles as done).

The draft is Start → Wait 300 s → Wait 200 s → Function returning {invoice: 42, settled: "after-waits"}. It runs through runCopilotLifecycle, the sim_cli handler, the embedded CLI, and the v2 execute and runs routes, against the worker's scripted probe (tools/probes/src/draft-run-slices.ts).

The case has three variants:

  • declared: the run detaches after 240 s with exit 4, continues as two workflows runs wait slices, and is read back with workflows runs get --include-output. No call holds the run (each under 250 s), no slice row is failed, and the model receives the run's real output.
  • undeclared: one held workflows run --manual with no --wait-timeout, exit 0, with the output.
  • own-budget: the model's own --wait-timeout 5 reaches it as exit 4, with the running receipt and the "Keep waiting" hint.

Two fixture changes make the controller cases runnable on a self-hosted default:

  • The harness pins direct Sim transport. The control fixture answers run control on its direct endpoint; under the checkpoint default, run control went unanswered and the turn timed out.
  • The fixture plan's execution timeout is set per case, so a run can outlast a slice.

Opt-in like the other controller cases: SIM_HELPERS_SMOKE=1, MSHIP_TEST_DATABASE_URL, MSHIP_WORKER_ROOT. Optionally MSHIP_PROBE_REDIS_URL, and MSHIP_SLICES_REPORT_PATH for the JSON evidence. It takes about nine minutes.

Companion: simstudioai/mothership#872

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: live test

Testing

Ran at this head against the worker branch feat/slice-draft-test-runs (70d213247), 09:55–10:04 PT. All three variants pass:

  • declared: 240.1 s (exit 4) → 240.0 s (exit 4) → 29.3 s (exit 0) → read-back 0.04 s; turn 509.9 s.
  • undeclared: one call, 260.4 s, exit 0.
  • own-budget: one call, 5.1 s, exit 4.

The case only adds a test and fixture knobs; no production code changes.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…l controller

A live case for the worker's sliced synchronous `workflows run`
(#9464, #9469): a draft with Wait blocks totalling 500 s runs
through runCopilotLifecycle, the sim_cli handler, the embedded CLI and the v2
execute and runs routes against the worker's scripted draft-run probe. With
workflow-run-wait-timeout declared it checks the detached first slice, the
`workflows runs wait` slices, the output read-back, that no call holds the run
and no slice row is failed, and that the model receives the run's output.
Without the capability it checks one held call as before, and a model's own
--wait-timeout 5 reaches the model as exit 4 with the running receipt.

The controller fixture now pins direct Sim transport, since self-hosted
defaults to checkpoint delivery and its run control went unanswered, and the
fixture plan's execution timeout is per case so a run can outlast a slice.
@vercel

vercel Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 30, 2026 5:21pm UTC

Request Review

@github-actions github-actions Bot added the requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep label Oct 30, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Cross-repo companion check

One or more companion PRs aren't merged into staging yet. Merging this without them will leave copilot and sim out of sync — merge them in lockstep.

  • ❌ simstudioai/mothership#872 — OPEN, not merged (targets staging) — feat(worker): wait on a synchronous workflows run in slices on a Sim that can de

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Low impact] The changes affect an opt-in live test, not production behavior.

Satisfy the repository's shared-helper and non-null assertion requirements before merging; the startup deadline is a non-blocking improvement.

Findings

  1. P2 Delay bypasses the shared helper ▶
  2. P2 Non-null assertion bypasses required checks ▶
  3. P2 Worker startup can stall ▶

Summary

Adds an opt-in live test that runs a draft workflow through the real controller.

  • The live test follows long draft runs through the real controller.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Real controller starts draft run] --> B{Capability declared?}
  B -->|No| C[Hold one call until completion]
  B -->|Yes| D{Model supplied wait budget?}
  D -->|Yes| E[Return running receipt after five seconds]
  D -->|No| F[Return running receipt after 240 seconds]
  F --> G[Wait in further slices]
  G --> H[Read completed run with output]
  C --> I[Check model received output]
  H --> I
Loading

Reviews (1) · Last reviewed commit: "test(mothership): drive a draft run past..." · Reviewed by Greptile

Uses sleep from @sim/utils/helpers, guards the last wait slice instead of a
non-null assertion, gives the probe's startup its own ten-second deadline like
the neighbouring controller case, and tells knip the probe path lives in the
worker checkout.
@waleedlatif1
waleedlatif1 merged commit 875f40a into staging Oct 30, 2026
67 checks passed
@waleedlatif1
waleedlatif1 deleted the test/live-sliced-draft-run branch October 30, 2026 17:36

This branch was successfully deployed

1 active deployment
Preview — c0c2b6c3 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-mothership-merge Has a companion PR on the mothership/copilot side — merge in lockstep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant