[earakely-scale]

PR 2 of the CUA env epic (FD-4345): M0.3, the reference driver (FD-4353), and M0.4, the conformance probe (FD-4354). It builds on the computer driver SDK from #336.

What's in it

• ComputerDriverEnvironment.declaration(platform) (protocol). The card a driver class serves on a platform, built without running the driver.
• The conformance probe (protocol, agentenv_protocol.computer.conformance). run_probe(url, level) and python -m agentenv_protocol.computer.conformance <url> check a running driver through its card, tools and extensions. The report has each check and a verdict per declared extension.
  • basic needs nothing on the computer: P0 readiness, P1 the card, P2 native frames, P5 refusal codes, and each declared extension.
  • full also stages the Fixture Desk, a Tk app, with data/add, starts it with computer-exec and reads back what reached it: P3 geometry, P4 click effects, P5 refusals change nothing, P6 keys, P7 scrolls and drags, P9 one unprivileged user, and the pointer, keyboard and data/add groups.
  • The probe removes everything it staged. P8 (leases) is not applicable, since no driver leases yet.
• The computer_driver env type (core, ComputerDriverEnv). Each env version stores the driver folder's files and what the class declares for each platform it supports.
  • module = "<file>:<class>" in env.toml names the class.
  • put reads the class in a subprocess (agentenv_protocol.computer._declare), where any module that isn't installed becomes an inert stub. Putting the reference driver therefore needs no X11 libraries.
  • Deploy is refused until agent-env can create a computer.
• The drivers bundle (core). It ships the reference driver as @local/agentenv-framework/drivers/reference, aliased reference. This is PR 1's X11 example moved out of examples/, now named by @computer_driver(name="reference").
• The test scaffold (tst/integration/computer/). A hand-written Ubuntu 24.04 image (Xvfb, the Fixture Desk, tini as PID 1) serves the reference driver as an MCP server on the server provider. agent-env run deploys it.
  • One task runs the probe at full, three times per run.
  • The other has Claude Code do the desk task through the computer_* tools. It runs only where a model endpoint is configured.
  • The scaffold goes away when agent-env composes computers.
• Docs. A conformance section in the protocol README, plus the env type, the bundle and the scaffold in AGENTS.md.

Where this differs from the design

1. Bundles with no tasks. check_bundle refused any installed bundle without tasks or evals, so the shipped drivers bundle failed it. A bundle with neither is now accepted when its aliases.toml names something for other bundles to use (bundle/plan.py). agent-env run drivers still says it has nothing to run.
2. Commits. The probe and its tests are one commit, not two. Two commits came out of testing, and three out of Greptile's review (see its threads): test(computer), so the protocol tests its own _declare in-process, and fix(protocol), which keeps the probe's data/add check inside its stage. Before that fix, a probe left ~/inbox/ behind and could delete a user's own ~/inbox/message.txt.
3. The deploy refusal says agent-env can't create a computer yet, rather than naming the milestone. Code carries no milestone names.
4. Core docs went into AGENTS.md and src/agent_env/drivers/README.md, since the core README is now the front door and the user guide lives on the website.
5. New core dependency. Core now depends on agentenv-framework-protocol[computer], because put imports the protocol's driver classes. Core already had mcp and uvicorn, so only Pillow is new.
6. Agent runs on amd64. The model key may be used only in-process on my laptop, never copied to the EC2 box, so the agent task ran on the laptop (arm64) only. Everything else ran on both.

Testing

• Unit tier: 10,862 passed, 64 skipped, 1 failed. The one failure is parse_test.py::TestFiles::test_deep_nesting[json] on my local Python 3.14 under xdist. It fails the same way on origin/main and passes alone; CI runs 3.12. Each commit was checked in a clean worktree: lint clean, and the unit tier gives the same result.
• Python 3.10: the protocol tests pass (801).
• Branch coverage (pytest-cov, by hand):
  • conformance/checks.py, client.py, report.py, __init__.py, platform.py and env/envs/computer_driver.py: 100%.
  • _declare.py 97% and conformance/__main__.py 95%: the uncovered line in each is the if __name__ == "__main__" guard.
  • driver.py: 99%. The uncovered line is PR 1's Linux exec default, which the e2e runs exercise.
  • conformance/fixture.py: 0% in unit tests, because it runs inside the computer. The e2e runs below exercise it.
• Integration: tst/integration/computer/scaffold_test.py passes both tests, the agent one with a model endpoint.

End-to-end and chaos at 2e38c6c, all on real X servers:

	EC2 c6i.4xlarge (amd64)	Laptop Docker (arm64)
agent-env run --task probe (scaffold_test.py, the probe at full three times)	passed, every check passed or n/a in all three runs (50 s)	passed
Probe CLI, basic	25 passed, P8 n/a	25 passed, P8 n/a
Probe CLI, full, three runs	48 passed, P8 n/a, each run	48 passed, P8 n/a
Home folder after the probe	unchanged	unchanged
Probe CLI, full, driver set to default_space = "norm"		48 passed, P8 n/a
PR 1's contract harness	58/58	58/58
Chaos	29/29	29/29
Agent desk task, Sonnet 5.5	not run (key)	passed 6/6 checks, 9 calls, 24 s
Agent desk task, Opus 5.5	not run (key)	passed 6/6 checks, 8 calls, 26 s

• In chaos, the X server killed under load recovers in 0.58 s on amd64 and 0.63 s on arm64. A timed-out command leaves no process running, because tini as PID 1 reaps the orphans PR 1's run left as zombies. Driver RSS goes from 67 to 81 MB after 500 frames on amd64, and from 65 to 79 MB on arm64, then holds through 700.
• The agents used only computer_* tools: one screenshot, five clicks, one type and one or two scrolls.

🤖 Generated with Claude Code

[earakely-scale]

End-to-end and chaos results at 2e38c6c

Every check here ran on a real X server.

• amd64: an EC2 c6i.4xlarge in agent-environments-admin, running Ubuntu 24.04 with Docker 29.9.
• arm64: Docker on my laptop.

On both, the image is the scaffold that tst/integration/computer/scaffold_test.py builds: Ubuntu 24.04, Xvfb 1280×800, the Fixture Desk, the reference driver, and tini as PID 1. The protocol is installed from this checkout.

Through agent-env (agent-env run of the scaffold bundle, deploy_env on the server provider, env_outcome_verifier):

• The probe task passes on both. Its verifier runs the probe at full three times, and every check passed or was not applicable in each run.
• The agent desk task passes on the laptop with Claude Code on both models; the six checks are read from the fixture's own state.
  • anthropic/claude-sonnet-5-5: 9 calls (1 screenshot, 5 clicks, 1 type, 2 scrolls), 24 s.
  • anthropic/claude-opus-5-5: 8 calls (1 scroll), 26 s.
  • Only computer_* tools were used.
• The agent task didn't run on EC2. The model key may only be used in-process on my laptop, never copied to the box.

The probe CLI against a running desktop:

• basic: 25 passed, P8 not applicable (no leasing).
• full: 48 passed, P8 not applicable, in each of three runs on each machine.
• With the driver set to default_space = "norm": 48 passed at full. This is Greptile's P1, fixed in aa0e449.
• After the probe, the home folder is the same as before, and the probe's own fixture process is gone. Before 8f0043f it left ~/inbox/ behind.

PR 1's contract harness: 58/58 on both machines, against the reference driver as shipped in the drivers bundle.

Chaos: 29/29 on both machines.

• X server killed under load: the first full success comes 0.58 s after the kill on amd64 and 0.63 s on arm64. No call hangs.
• Driver restarted: old frames are refused stale_frame.
• Flood, fuzz, network: the flood (64/64 clicks land), the fuzz (1,000 malformed calls) and netem (150 ms ± 50 ms, 3% loss) all pass.
• Data plane and exec: the data-plane races and escapes, and the exec timeout, caps and idempotency, pass.
• Orphans: a timed-out command leaves no process running ("sleeps left running: 0"). tini reaps the orphans that PR 1's run left as zombies.
• Memory: driver RSS goes from 67 to 81 MB after 500 frames on amd64 and from 65 to 79 MB on arm64, then holds through 700.

Fixed on this branch because of these runs

• 8f0043f: the probe's data/add check wrote ~/inbox/message.txt, outside its stage. It now writes <stage>/state/inbox.txt, which goes with the stage.

🤖 Generated with Claude Code