Skip to content

Enforce Go coverage floors before reporting CI success #1044

Open

Description

@tbhb-agent

Problem

The coverage check can succeed after Code Quality rejects its upload, while CI does not enforce the Go coverage floors. PR #1033 at head fe76797652eddb1a54ad7990139dc03f56d3e82e has a successful coverage job, but its combined upload at 2026-11-01 07:02:27 UTC returned HTTP 403: "Code quality is not enabled for this repository. Enable code quality in your repository settings." The upload step and job both concluded success.

Both earlier PRs merged with the same unproven floor enforcement:

All three PRs were authored by tbhb-agent from branches in this repository, not forks. All three coverage logs report GITHUB_TOKEN permission CodeQuality: write, and the upload eligibility step succeeds. The logged cause is disabled repository Code Quality, not a demonstrated missing token scope, fork restriction, or App identity restriction. Repository settings and any deeper service-side cause were not independently confirmed; these logs do not establish a restriction specific to agent authors.

Source inspected on fetched origin/main at 37f71fd4f085d8354eb6bd1803b57c4c0385830c explains the green result:

  • .github/workflows/ci.yml:213 runs cover-go-binary, and .github/workflows/ci.yml:281 runs cover-go-merge. The upload inputs at .github/workflows/ci.yml:237 and .github/workflows/ci.yml:292 set fail-on-error: false.
  • .devtools/tasks/go.toml:200 generates coverage data and XML; .devtools/tasks/go.toml:214 merges and renders it. Neither task invokes the floor checker. .devtools/tasks/go.toml:228 defines that separate checker, exposed at mise.toml:695, but the CI workflow does not invoke it.
  • .testcoverage.yml:12 requires independently enforced floors of 75% per file, 80% per package, and 85% total, subject to its existing exclusions.

The claim that CI did not measure coverage at all is too broad: the test logs contain measured package percentages, and the merge step succeeds. What is not demonstrated is enforcement of the three configured floors or successful publication. The Ubuntu test upload also returns the same 403 on each PR. No claim is made that any of these heads actually falls below a floor, or that no local check ran before merging.

Acceptance criteria

  1. AC1: When coverage publication fails, the coverage result fails unless CI independently measures and enforces all three configured floors for the same PR head; a green result includes visible evidence of that enforcement.
  2. AC2: On agent-authored PRs, a file below 75% coverage causes the coverage result to fail even when package and total floors pass; a file at 75% meets the file floor.
  3. AC3: On agent-authored PRs, a package below 80% coverage causes the coverage result to fail even when file and total floors pass; a package at 80% meets the package floor.
  4. AC4: On agent-authored PRs, total coverage below 85% causes the coverage result to fail even when file and package floors pass; total coverage at 85% meets the total floor.
  5. AC5: Non-functional: Existing required checks, token permission limits, coverage thresholds, and coverage exclusions are preserved without weakening a gate.

Scope and non-goals

Restore truthful coverage status and enforce the existing Go floors in CI, including agent-authored PRs. The fix is separate work and is not part of PR #1033. The expected fix touches .github/workflows/ci.yml and may use the existing coverage tasks.

Non-goal: Change the three floors, expand exclusions, modify PR #1033, or infer that enabling Code Quality alone enforces .testcoverage.yml.

Approach

Make successful coverage status depend on actual floor enforcement for the tested head. Treat an upload failure as a failed result unless the floors are independently measured and enforced. Use the existing checker where suitable, retain visible measured results, and verify behavior with rejected uploads and isolated floor violations. Resolve publication only within the permissions already granted; do not assume agent identity caused the 403.

Dependencies

None.

Size

S

Verify plan

  • AC1: Exercise a rejected upload, including HTTP 403, for a known PR head. Confirm a failed coverage result, or inspect successful independent enforcement of all three floors and its visible measured results for that same head. Confirm publication alone cannot substitute for floor enforcement.
  • AC2: On an agent-authored PR, use profiles with file coverage immediately below and at 75%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
  • AC3: On an agent-authored PR, use profiles with package coverage immediately below and at 80%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
  • AC4: On an agent-authored PR, use profiles with total coverage immediately below and at 85%, keeping other floors satisfied. Confirm rejection below the boundary and acceptance at the boundary.
  • AC5: Compare the resulting workflow, permissions, required checks, thresholds, and exclusions against the base. Confirm no permission expansion, skipped required check, lowered floor, or expanded exclusion.

Risks

Enforcing an existing floor may expose previously hidden failures; resolve those through tests rather than lowering the floor. Upload failures may block otherwise passing changes if publication is the selected gate. The fix loosens no gate. Undo: revert the dedicated fix commit if it breaks CI, preserving the defect report and treating coverage as unproven until a corrected fix passes verification.

Operator authority

owner-only GitHub action: The expected fix touches .github/workflows/ci.yml, an owner-tier path, and requires the owner's approval of the exact PR head before merge. Enabling repository Code Quality, if chosen, also requires the owner's repository-setting authority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    None yet

    Severity

    None yet

    Work type

    None yet

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions