Repository navigation
Enforce Go coverage floors before reporting CI success #1044
Description
Problem
The coverage check can succeed after Code Quality rejects its upload, while CI does not enforce the Go coverage floors. PR #1033 at head fe76797652eddb1a54ad7990139dc03f56d3e82e has a successful coverage job, but its combined upload at 2026-11-01 07:02:27 UTC returned HTTP 403: "Code quality is not enabled for this repository. Enable code quality in your repository settings." The upload step and job both concluded success.
Both earlier PRs merged with the same unproven floor enforcement:
- PR fix(ghpost): print GitHub's error after a failed write #1028, head
3869284caf368542659b8c0a63b6002d730ef319, had a successful coverage job whose combined upload returned the same 403 at 2026-11-01 01:51:57 UTC. It merged at 02:08:13 UTC as3416703da9ced3857d249012f7efafc672ce7b17. - PR feat(agentrepo): add lint-types #1023, head
f0d36d63ece339a7b9d65e56b4a069be45e796e9, had a successful coverage job whose combined upload returned the same 403 at 2026-11-01 04:16:58 UTC. It merged at 04:35:53 UTC as37f71fd4f085d8354eb6bd1803b57c4c0385830c.
All three PRs were authored by tbhb-agent from branches in this repository, not forks. All three coverage logs report GITHUB_TOKEN permission CodeQuality: write, and the upload eligibility step succeeds. The logged cause is disabled repository Code Quality, not a demonstrated missing token scope, fork restriction, or App identity restriction. Repository settings and any deeper service-side cause were not independently confirmed; these logs do not establish a restriction specific to agent authors.
Source inspected on fetched origin/main at 37f71fd4f085d8354eb6bd1803b57c4c0385830c explains the green result:
.github/workflows/ci.yml:213runscover-go-binary, and.github/workflows/ci.yml:281runscover-go-merge. The upload inputs at.github/workflows/ci.yml:237and.github/workflows/ci.yml:292setfail-on-error: false..devtools/tasks/go.toml:200generates coverage data and XML;.devtools/tasks/go.toml:214merges and renders it. Neither task invokes the floor checker..devtools/tasks/go.toml:228defines that separate checker, exposed atmise.toml:695, but the CI workflow does not invoke it..testcoverage.yml:12requires independently enforced floors of 75% per file, 80% per package, and 85% total, subject to its existing exclusions.
The claim that CI did not measure coverage at all is too broad: the test logs contain measured package percentages, and the merge step succeeds. What is not demonstrated is enforcement of the three configured floors or successful publication. The Ubuntu test upload also returns the same 403 on each PR. No claim is made that any of these heads actually falls below a floor, or that no local check ran before merging.
Acceptance criteria
- AC1: When coverage publication fails, the coverage result fails unless CI independently measures and enforces all three configured floors for the same PR head; a green result includes visible evidence of that enforcement.
- AC2: On agent-authored PRs, a file below 75% coverage causes the coverage result to fail even when package and total floors pass; a file at 75% meets the file floor.
- AC3: On agent-authored PRs, a package below 80% coverage causes the coverage result to fail even when file and total floors pass; a package at 80% meets the package floor.
- AC4: On agent-authored PRs, total coverage below 85% causes the coverage result to fail even when file and package floors pass; total coverage at 85% meets the total floor.
- AC5: Non-functional: Existing required checks, token permission limits, coverage thresholds, and coverage exclusions are preserved without weakening a gate.
Scope and non-goals
Restore truthful coverage status and enforce the existing Go floors in CI, including agent-authored PRs. The fix is separate work and is not part of PR #1033. The expected fix touches .github/workflows/ci.yml and may use the existing coverage tasks.
Non-goal: Change the three floors, expand exclusions, modify PR #1033, or infer that enabling Code Quality alone enforces .testcoverage.yml.
Approach
Make successful coverage status depend on actual floor enforcement for the tested head. Treat an upload failure as a failed result unless the floors are independently measured and enforced. Use the existing checker where suitable, retain visible measured results, and verify behavior with rejected uploads and isolated floor violations. Resolve publication only within the permissions already granted; do not assume agent identity caused the 403.
Dependencies
None.
Size
S
Verify plan
- AC1: Exercise a rejected upload, including HTTP 403, for a known PR head. Confirm a failed coverage result, or inspect successful independent enforcement of all three floors and its visible measured results for that same head. Confirm publication alone cannot substitute for floor enforcement.
- AC2: On an agent-authored PR, use profiles with file coverage immediately below and at 75%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
- AC3: On an agent-authored PR, use profiles with package coverage immediately below and at 80%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
- AC4: On an agent-authored PR, use profiles with total coverage immediately below and at 85%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
- AC5: Compare the resulting workflow, permissions, required checks, thresholds, and exclusions against the base. Confirm no permission expansion, skipped required check, lowered floor, or expanded exclusion.
Risks
Enforcing an existing floor may expose previously hidden failures; resolve those through tests rather than lowering the floor. Upload failures may block otherwise passing changes if publication is the selected gate. The fix loosens no gate. Undo: revert the dedicated fix commit if it breaks CI, preserving the defect report and treating coverage as unproven until a corrected fix passes verification.
Operator authority
owner-only GitHub action: The expected fix touches .github/workflows/ci.yml, an owner-tier path, and requires the owner's approval of the exact PR head before merge. Enabling repository Code Quality, if chosen, also requires the owner's repository-setting authority.
Activity