Repository navigation
simplify(diagnostics): share daemon compiler admission and bounds - #4546
ScriptedAlchemy wants to merge 19 commits into
Conversation
Replace the token-file task-lock protocol (random ownership token, hard-link publication, staging files, PID/age reclamation, retrying destructor unlink) with a nonblocking exclusive FileLease on a stable `<key>.lease` inode held for the task and retained-settlement lifetime. The lease file is never unlinked, so exclusion cannot split across inodes. A live pre-lease holder (`<key>.lock` naming a live pid) still keeps the key; dead or pid-reused owners' artifacts are retired and unverifiable ownership is a typed error. The stale-lock timeout no longer applies. Refs #4426 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
|
Independent source review at exact head
A separate correction worker is now implementing actual compiler supervision and project-owned terminal settlement with deterministic regressions. The dependent #4440 publication is held until the correction is reviewed and verified. No timeout increase, higher capacity, clean empty result, or branch rewrite is proposed. The existing local test report is preserved, but does not discharge these untested lifetime boundaries. |
Fleet logs show five complete 48,000-function generations take 53–65 seconds each. The default medium deadline kills the final refresh near 300 seconds despite steady successful publication. Declare the measured aggregate workload through the existing Cargo-to-Bazel size metadata, preserving every refresh, memory bound, and assertion. The current #4542 run independently confirmed the same limit: 170/171 tests passed; complete generations published at 62, 123, 184, and 245 seconds before the final refresh was interrupted at 300.1 seconds with nominal memory usage. Failure evidence: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/40248084891/job/121393230765
Resolve .github/workflows/ci.yml to master's ci-windows configuration, which already carries this branch's --jobs=2 Windows concurrency bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
64b873e to
7f0d09f
Compare
The observer loop read the stop flag after accept() returned WouldBlock. A connection the action completed between that accept and the flag load was never counted, so a loaded runner saw 0 instead of 1. Read the flag before accept(): a WouldBlock after a set flag proves the backlog was already empty once the action returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(test): drain queued connections before HTTP observation stops
…k-lease simplify(automation): hold one lifetime file lease per task key
a449b1d to
5a46a14
Compare
Route eviction can retain a project's Work runtime while a sibling commits new configuration. Reopening the route previously refused that same store authority before the compiler producer could register. Refresh the policy, topology, routing and evidence admission together under the captured runtime and canonical current-configuration fence. Keep exact store, actor, scope, grant and Work authority checks. Refresh the retained fan-out recovery view without replacing recovery owners or their signal. Extend the real compiler route journey to verify stable grant identity, current Work authority, stale and sibling rejection, and retained authority after refusals. Preserve all existing assertions and budgets. Validation: pinned rustfmt and diff checks passed; independently reviewed. Native tests were not run locally because of disk and memory constraints. Hosted CI must verify the repaired route journey.
The pending-producer fixture deliberately holds a compiler CPU admission until the test releases it. Waiting for lexical readiness before that release can leave the sorter waiting for the same process-wide CPU width. The source-proven cycle is consistent with the failing CI snapshot: four files were built in 60 ms before setup stalled at lexical publication. This is not an occupancy measurement or a reason to raise a deadline. Use the existing seeded production opener, prove that the held compiler actually started, and preserve the typed pending-state assertions. Release the process before requiring graph readiness, then require its published TS4023 finding. Keep the fixture workload and all budgets unchanged. Validation: pinned rustfmt and diff checks passed; independently reviewed. Native tests were not run locally because of disk and memory constraints. Hosted CI must verify the complete pending-to-published journey.
Hold the canonical configuration writer lane while validating and publishing a changed retained Work runtime. Refuse stale candidates without replacing the incumbent or waiting for a new owner. Exercise the real route journey with a newer durable configuration committed while refresh is paused, then admit the current revision.
Why
Each producer had its own four-child compiler pool and effectively unlimited subprocess capture. Concurrent project producers did not share their child capacity.
What changed
Scope
Fixes #4439. Parent link is #4545 on
simplify/diagnostics-project-compiler-status.The project-owned status cutover remains intact. Compiler-owned configuration evidence in #4440 and operation retirement in #4403 remain separate links. This change does not modify #4524.
Tradeoffs
The existing background CPU authority meters active compiler work. A daemon-owned semaphore additionally bounds live compiler children without another scheduler or a process-global registry. No RSS reduction or production budget calibration is claimed.
Blast Radius
Automatic project-pinned TypeScript diagnostics now share daemon admission. A stalled or excessive-output compiler fails rather than publishing clean evidence. Retrieval still reads generation-bound publications without running a compiler.
Verification
Held ASK
Devin thread on
scheduler.rsstale-lock admission (stale_lock_secsvs unverifiable owner) is a concurrency/lock-admission question. Left open for the operator: remove the obsolete setting, or carry the configured bound into the legacy-owner decision. Not implemented here.Current-head CI failure repairs
The Linux fleet run 40437266717 completed 169 of 171 targets successfully. This follow-up repairs the two failed journeys without changing their budgets or dropping existing assertions.
The retained-route journey reopened a project after a sibling changed configuration. The old Work runtime rejected the same store authority before the compiler producer could register. Registration now refreshes policy, topology, routing, and evidence admission together, fenced by the captured incumbent and the canonical current configuration. Exact store, actor, grant, scope, and Work authority remain required. The retained fan-out recovery view receives the same admission update without replacing recovery owners or creating an owner cycle. The real journey now also checks stable grant identity, the current registration, refusal of stale or sibling registrations, and preservation of the current authority after refusal.
The pending-compiler fixture held its real compiler until the test could release it, but setup waited for lexical publication first. Source inspection establishes a reachable CPU admission/readiness cycle consistent with the small four-file CI fixture stalling after index build; this is not a measured occupancy claim. The fixture now uses the existing production opener, proves compiler startup before the unchanged typed pending assertions, releases the compiler, then requires graph readiness and the published
TS4023finding. Workload, polling bounds, and existing result checks are preserved.Both focused commits passed pinned formatting, whitespace checks, normal pre-commit/commit-msg hooks, and independent source review. The complete native journeys were not run locally because of disk and memory constraints; the new head must pass hosted CI.
Configuration publication race follow-up
A changed retained Work runtime now verifies and publishes its configuration under the configuration store's existing canonical writer lane. The store reads durable current state through a registered immediate transaction, revalidates its retained authority, and invokes a synchronous registry transition before releasing the lane. The captured-incumbent comparison remains in place to prevent a delayed registrar from replacing a newer installed runtime.
The registry transition only reconciles an existing component; missing, reserved, closed, or fenced slots are refused. It performs no new owner build or asynchronous wait. Complete routing and recovery admission are refreshed together. The read-only transaction is dropped synchronously after the callback, using the existing rollback-before-writer-release contract, with no post-publication await.
Initial and unchanged retained pins preserve their existing lifetime. Work settings retain their daemon-restart semantics: a durable commit after successful publication is allowed, while a superseded candidate cannot claim to be current at publication.
The existing real route-eviction/compiler-status journey now pauses candidate B, commits C through the actual configuration surface, resumes B and requires its typed refusal with installed A unchanged, then installs and verifies C. Existing assertions and budgets remain intact. Pinned formatting, whitespace checks, normal commit hooks, and independent source/authority/lock-order review passed. The complete native journey was not executed locally and must pass hosted CI.