[zahin-mohammad]

Description

The weekly iyarc-prune workflow removes obsolete vulnerability-scan exceptions. It could finish with validated dependency edits but no commit or pull request because the agent treated a checkout on master as a local run. Reported failure and acceptance criteria.

• Give the workflow explicit ownership of publication: require successful agent completion, reject any change to the original checkout commit, check tracked and untracked files plus a nonempty validation report, preserve existing labels, skip an existing open prune pull request, then create a signed commit and a non-draft pull request against master with the existing labels and assignment. Fail when publication produces no pull request URL. Workflow.

• Have the agent leave validated edits uncommitted and write its report to the workflow-provided temporary path. Local runs continue to leave edits for developer review. Agent instructions.

• Replace the unsupported scanner severity flag with the release workflow's scanner output and severity evaluation. Audit instructions.

• Install scanner version 2.3.8 before the agent runs, with a pinned SHA-256 checksum (a digest used to detect changed downloads). Scanner setup.

WARNING: Generated pull requests use GitHub's repository-scoped workflow credential (GITHUB_TOKEN). Their automated checks may need a maintainer to select Approve workflows to run before they start. The agent report and workflow summary now state this requirement. Current GitHub behavior.

Issue Number

WAL-2232

Type of change

• Bug fix
• Agent instruction update

How Has This Been Tested?

VERIFIED locally: workflow syntax, formatting, commit rules, and 34 execution, publication, label, and scanner-installation cases. The decision harness executes the actual workflow Bash steps in an isolated temporary Git repository, with the GitHub command-line client replaced by a controlled stub.

Validation commands and actual output

Run from the repository root:

nix shell github:NixOS/nixpkgs/e9b7fc70551fcd2e52077d58d2c3bcee2428cf8d#actionlint github:NixOS/nixpkgs/e9b7fc70551fcd2e52077d58d2c3bcee2428cf8d#shellcheck -c actionlint .github/workflows/iyarc-prune.yml

Exit status 0; no workflow diagnostics. The tool emitted these environment warnings:

warning: unknown setting 'eval-cores'
warning: unknown setting 'lazy-trees'

node_modules/.bin/prettier --check .claude/agents/iyarc-prune.md .github/workflows/iyarc-prune.yml

Checking formatting...
All matched files use Prettier code style!

nix shell github:NixOS/nixpkgs/e9b7fc70551fcd2e52077d58d2c3bcee2428cf8d#coreutils -c node /tmp/wal-2232-validate.cjs

PASS successful array
PASS successful object
PASS agent error
PASS turn limit
PASS missing result
PASS missing success flag
PASS malformed execution
PASS empty execution
PASS missing execution file
PASS no changes
PASS validated changes
PASS staged changes
PASS unexpected tracked path
PASS unexpected untracked path
PASS new workspace manifest
PASS committed edits
PASS committed unexpected path
PASS empty agent commit
PASS missing checkout commit
PASS manifest without exclusions
PASS missing report
PASS empty report
PASS existing open pull request
PASS pull request query failure
PASS missing labels
PASS one missing label
PASS mixed case existing labels
PASS label query failure
PASS label creation failure
PASS publication URL present
PASS publication URL missing
PASS scanner installation exports path
PASS scanner checksum mismatch
PASS scanner download failure
Workflow decision checks: 34/34 PASS

Scanner-installation checks use a local executable fixture and stub downloads; they verify path export and rejection of download/checksum failures. Separately, the real Linux release binary matched the published checksum:

printf '%s  %s\n' bc98e15319ed0d515e3f9235287ba53cdc5535d576d24fd573978ecfe9ab92dc /tmp/wal-2232-scanner-check/osv-scanner | shasum -a 256 -c

/tmp/wal-2232-scanner-check/osv-scanner: OK

The temporary harness is at /tmp/wal-2232-validate.cjs; it is not checked in. Its load-bearing execution and assertion excerpts are:

const steps = yaml.load(fs.readFileSync('.github/workflows/iyarc-prune.yml', 'utf8')).jobs.prune.steps;
const script = (name) => steps.find((step) => step.name === name).run;

const result = run('bash', ['-e', '-o', 'pipefail', '-c', script(stepName)], env);

assert.equal(result.status === 0, shouldPass, `${name}: ${result.stdout}\n${result.stderr}`);
assert.equal(result.output.includes('ready=true'), ready, name);

yarn check-commits

Output excerpt:

⧗   input: fix(root): guard prune publication and provision scanner

Ticket: WAL-2232
✔   found 0 problems, 0 warnings

git diff --check and the repository commit hooks also passed.

Unverified live behavior: the local harness does not exercise model inference, Linux scanner execution, GitHub token permissions, or signed publication against this repository. The repository setting Allow GitHub Actions to create and approve pull requests also needs verification: gh api repos/BitGo/BitGoJS/actions/permissions/workflow returned HTTP 403 (Resource not accessible by personal access token). After merge, manually dispatch the workflow and verify the resulting commit signature, pull request, labels, assignment, and required checks.

Checklist

• Reviewed both changed files
• Workflow syntax and publication decision checks passed locally
• Agent instructions updated
• Signed commit follows Conventional Commits and references WAL-2232

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[linear-code]

WAL-2232

[zahin-mohammad]

@claude review always

[github-actions]

Claude finished @zahin-mohammad's task in 26s —— View job

---

Review of PR #10087

The approach is sound. The workflow now owns publication, and the agent only leaves uncommitted edits plus a report. I found no blocking bugs, but there are a few things to check.

Items to check

1. CI won't run on the created PR (.github/workflows/iyarc-prune.yml, "Create signed dependency pull request"). peter-evans/create-pull-request with GITHUB_TOKEN creates PRs that do not trigger pull_request workflows. The agent instructions treat PR CI as the "backstop" for the full test suite, so these PRs would sit without checks. If branch protection requires checks, they will be unmergeable until someone pushes an empty commit or closes and reopens the PR. Consider using a GitHub App token or PAT, or call this out in the PR description.

2. Repo setting dependency. Creating PRs from GITHUB_TOKEN needs "Allow GitHub Actions to create and approve pull requests" enabled for the repo or org. The PR notes that live behavior is unverified, so check this setting before the first dispatch.

3. Pinned SHA and ref-pin lint. lint-gha.yaml lists peter-evans/create-pull-request as ref-pin. Confirm that 22a9089… is the v7 tag commit and that the lint job passes with it.

4. Silent early exits. In "Check changes are ready", an existing open prune PR exits 0 with only a step-summary line. That is fine. But labels are created only after the dedupe check, and gh label create --force failing would fail the job after the agent has already spent its tokens. This is acceptable, but you could make it || true, since the labels are a convenience.

5. Changed-file allowlist. git diff --name-only HEAD ignores untracked files, so a stray new file would not be caught. It also would not be committed, because add-paths limits what is staged, so this is harmless. The modules/*/package.json allowance is broader than the agent's "Only modify osv-scanner.toml, package.json, yarn.lock" rule. That is probably intended, since yarn upgrade-dep touches workspaces, but you may want to align the wording in the agent doc.

6. Concurrent runs. The branch name osv-scanner-prune/${{ github.run_id }} is unique per run. The concurrency group plus the open-PR check prevents duplicates. This is fine.

7. Minor. The "Require successful Claude completion" step still runs if: always() and exits non-zero on a missing result. That is the desired fail-closed behavior. The unrelated Prettier blank-line additions in the agent markdown are harmless.

Summary: the logic reads correctly. The main follow-up is item 1 (CI not triggering on bot-created PRs) and verifying items 2 and 3 before the first manual dispatch. I did not run anything locally. This was a read-only review.
· Branch: codex/WAL-2232-iyarc-prune-create-pr

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

[claude]

🟡 (optional) If the agent commits its validated edits locally, the job passes green and publishes no PR. That is the silent no-PR outcome this change set out to fix. The readiness check runs git diff --name-only HEAD, so committed edits look like "No dependency changes to publish." and the step exits 0. Fix: record the checkout SHA (or use origin/master) and diff the working tree against it. Fail when git log <base>..HEAD is non-empty, so a local commit cannot hide edits from the allow-list and the PR-body checks.

Why this was flagged

The trigger is the agent running git commit despite the new "do not commit" instruction in .claude/agents/iyarc-prune.md. Its earlier prompt told it to commit, and it still has Bash with git and GH_TOKEN. Then git diff --name-only HEAD at iyarc-prune.yml:166 is empty. The step takes the exit-0 branch at :167-170, ready is never set, and both the create-pull-request step and the "Report pull request" URL check are skipped. The job ends green with validated dependency fixes and no pull request, and the allow-list, osv-scanner.toml and PR-body checks never run on the committed changes. Nothing else catches it: the "Require successful Claude completion" step only checks the result subtype, not where the edits ended up. Comparing against the SHA recorded at checkout (ref: master, :29) instead of the moving HEAD would close this.

Verification: Trigger: the agent runs git commit even though .claude/agents/iyarc-prune.md:131-132 tells it "Do not create a branch, commit, push, or open a pull request yourself". The readiness step runs CHANGED_FILES="$(git diff --name-only HEAD)", which only sees uncommitted changes, so it exits 0 without writing ready=true. Nothing compares HEAD with origin/master, so the job ends green with no PR.

[zahin-mohammad]

Fixed in 70c3b4a. The readiness step now uses the checkout action's immutable commit output, fails if the agent moved HEAD, and checks tracked plus untracked files against that base. This prevents an agent-created commit from turning validated edits into a successful no-change result. Updated guard.

The temporary decision harness covers committed edits, a committed unexpected path, an empty agent commit, a missing base, and untracked paths. All 34 cases passed; invocation and actual output are in the PR description.

[claude]

🟡 nit (optional): Every run that publishes a PR gives the shared automated, dependencies and security labels a new random color. gh label create with --force but no --color picks a random color and updates any label that already exists. The base prompt only created labels that were missing. Fix: drop the loop, since GitHub creates missing labels when create-pull-request applies them, or create only labels that are missing, or pass a fixed --color.

Why this was flagged

The trigger is any run where the "Check changes are ready" step reaches iyarc-prune.yml:189-191, which is every run that goes on to publish. The dependencies and security labels are repo-wide and also used by other PRs. gh label create picks a random color when --color is omitted, and --force turns an "already exists" error into an update of the existing label, so its color changes on each publishing run. On the base branch the agent was told to create only missing labels ("create any missing one with gh label create --force"). The loop is also redundant, because create-pull-request's labels: input adds labels through the issues API, which creates missing labels.

Verification: Nit. The new code at .github/workflows/iyarc-prune.yml:189-191 runs gh label create "$label" --repo "$GITHUB_REPOSITORY" --force for every label, with no check for whether it already exists. It passes no --color. So on each publishing run the existing repo-wide automated, dependencies and security labels get a new color. The workflow now does it unconditionally, so this churn is new with this change.

[zahin-mohammad]

Fixed in 70c3b4a. The workflow lists existing labels, compares names without regard to case, and creates only missing labels. It no longer uses --force, so existing label colors are preserved. Updated label handling.

The local harness verified zero creation calls when all labels exist, creation of only the missing label, mixed-case existing labels, and failures from label lookup/creation. Results are in the PR description.

[claude]

🟣 pre-existing, not blocking: Pre-existing: the prune PR is opened with GITHUB_TOKEN, so PR CI never runs on it. The prompt treats that CI as its backstop, and the PR sits with no required checks. GitHub does not start pull_request workflows for events caused by GITHUB_TOKEN. That rules out .github/workflows/ci.yml and every other PR workflow. Fix: give create-pull-request a GitHub App installation token or a PAT that can trigger workflows. Keep sign-commits: true, which needs the same token to sign through the API.
A small fix can ride a push you are already making; otherwise a short reply is enough.

Why this was flagged

The scheduled run validates a removal. The "Create signed dependency pull request" step at .github/workflows/iyarc-prune.yml:194-216 then pushes the branch and opens the PR with token: ${{ secrets.GITHUB_TOKEN }} (line 199). GitHub suppresses workflow triggers for events made with GITHUB_TOKEN. So .github/workflows/ci.yml, whose on: pull_request: branches: [master] is at lines 6-8, never runs on the PR. The PR shows no status checks and branch protection blocks the merge until someone pushes a new commit or closes and reopens the PR with another identity. The prompt still says at .claude/agents/iyarc-prune.md:127 that "The full test suite still runs in PR CI as a backstop". That is false, and the agent only built and tested the affected modules. The base had the same problem, because gh pr create and the commit_files MCP tool also used GITHUB_TOKEN. This rewrite of the publication step is the place to fix it.

Verification: Pre-existing. The new "Create signed dependency pull request" step (.github/workflows/iyarc-prune.yml) runs with token: ${{ secrets.GITHUB_TOKEN }}, so the pull_request trigger in .github/workflows/ci.yml (lines 6-9) never fires for this PR. .claude/agents/iyarc-prune.md:127 says "The full test suite still runs in PR CI as a backstop". The base also opened the PR with GITHUB_TOKEN (base iyarc-prune.md:150).

[zahin-mohammad]

Corrected the unconditional automated-checks promise in 70c3b4a. Current GitHub documentation says pull requests opened or updated with GITHUB_TOKEN create workflow runs that require maintainer approval; the older statement that these events never create runs no longer matches the documented behavior. GitHub documentation.

The agent instructions, requested PR body, workflow summary, and this PR's warning now tell maintainers to approve pending runs and verify required checks before merging. Agent instructions, workflow summary.

The existing token and API-based commit signing remain in place. Live behavior and the repository permission setting remain explicit post-merge verification items; the settings endpoint returned HTTP 403 with my available credentials.

[claude]

🟣 pre-existing, not blocking: Pre-existing: on the CI runner the agent is told to run osv-scanner v2.3.8, but no workflow step installs it. Validation may fail, so no exclusion gets pruned. The release workflows use google/osv-scanner-action (a container action) to get the scanner. iyarc-prune.yml has no matching setup step, and osv-scanner is not preinstalled on ubuntu-latest. Fix: add a step to .github/workflows/iyarc-prune.yml that installs osv-scanner v2.3.8 (a pinned release binary with a checksum) before the Claude step. Then the agent prompt can rely on that binary.
A small fix can ride a push you are already making; otherwise a short reply is enough.

Why this was flagged

The scheduled run reaches the audit gate, where .claude/agents/iyarc-prune.md:27-30 tells the agent to run osv-scanner --config=osv-scanner.toml --format=json .... .github/workflows/iyarc-prune.yml only sets up Node, Corepack and AWS before the Claude step. Release workflows get the scanner from google/osv-scanner-action/osv-scanner-action@ 9a49870... (.github/workflows/npmjs-release.yml:275, publish.yml:61), and nothing equivalent exists here. If the agent cannot install the exact version itself, the command is missing or the version differs from the release gate. The prompt then requires the agent to fail validation and revert, and no PR body is written. The weekly job keeps producing no PR, the failure the PR says it fixes. The base also lacked an install step, but this diff newly pins the agent to v2.3.8 and the release gate's output.

Verification: Pre-existing. .github/workflows/iyarc-prune.yml has no step before "Run Claude Code (Bedrock)" that installs osv-scanner, unlike npmjs-release.yml:275 and publish.yml:61. The prompt at .claude/agents/iyarc-prune.md:27-29 tells the agent to run osv-scanner. If the binary is missing, validation must fail, every exclusion is restored, and nothing is published. The base prompt had the same gap.

[zahin-mohammad]

Fixed in 70c3b4a. A setup step now downloads the Linux version 2.3.8 scanner, verifies its pinned SHA-256 checksum, checks its version, and exports its directory before the agent runs. Scanner setup.

The downloaded Linux binary matched the published release checksum. The local installation harness used a fixture executable to verify path export and rejection of checksum/download failures; actual output is in the PR description. Linux execution remains part of the first live workflow verification.