Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ on:
schedule:
- cron: "42 6 * * 0"

# New pushes cancel older runs for the same pull request. Each job queues by
# workflow, job, and author, so one author runs one copy of each job at a time.
# GitHub orders jobs by when they start waiting, but does not guarantee FIFO.
# Each queue holds up to 100 pending jobs; later arrivals are canceled.
# Timeouts release queues after hung jobs. Bots and non-PR events use unique
# run IDs and do not queue together.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

jobs:
analyze:
name: Analyze (${{ matrix.language }})
Expand All @@ -28,6 +38,10 @@ jobs:
# - https://gh.io/using-larger-runners (GitHub.com only)
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
timeout-minutes: 45
concurrency:
group: ${{ github.workflow }}-analyze-${{ matrix.language }}-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
permissions:
# required for all workflows
security-events: write
Expand Down
51 changes: 50 additions & 1 deletion .github/workflows/format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,24 @@ on:
permissions:
contents: read

# New pushes cancel older runs for the same pull request. Each job queues by
# workflow, job, and author, so one author runs one copy of each job at a time.
# GitHub orders jobs by when they start waiting, but does not guarantee FIFO.
# Each queue holds up to 100 pending jobs; later arrivals are canceled.
# Timeouts release queues after hung jobs. Bots and non-PR events use unique
# run IDs and do not queue together.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

jobs:
check-claude-md-symlink:
name: CLAUDE.md symlink guard
runs-on: ubuntu-latest
timeout-minutes: 5
concurrency:
group: ${{ github.workflow }}-check-claude-md-symlink-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -42,9 +56,22 @@ jobs:
check-ci-node-pin:
name: CI Node pin guard
runs-on: ubuntu-latest
timeout-minutes: 5
concurrency:
group: ${{ github.workflow }}-check-ci-node-pin-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

- name: Install yq
shell: bash
run: |
curl --fail --location --silent --show-error \
https://github.com/mikefarah/yq/releases/download/v4.54.1/yq_linux_amd64 \
--output "$RUNNER_TEMP/yq"
chmod +x "$RUNNER_TEMP/yq"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"

# The guard's tests check the tree before the guard landed; fetch only
# that commit instead of the full history.
- name: Fetch the pre-guard commit
Expand All @@ -55,17 +82,27 @@ jobs:
with:
node-version-file: .tool-versions

- name: Test the guard
- name: Test the Node pin guard
run: node --test scripts/check-ci-node-pin.test.mjs

# Every job that compiles trusted-server-core also runs the TSJS build, so
# it must set up the `.tool-versions` Node first (see issue #1204).
- name: Verify jobs pin Node before compiling Rust
run: node scripts/check-ci-node-pin.mjs

- name: Test the CI concurrency policy guard
run: node --test scripts/check-ci-concurrency.test.mjs

- name: Require per-PR cancellation, per-author job queues and timeouts
run: scripts/check-ci-concurrency.sh

format-rust:
name: cargo fmt
runs-on: ubuntu-latest
timeout-minutes: 25
concurrency:
group: ${{ github.workflow }}-format-rust-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4
Comment thread
dhruv8sh marked this conversation as resolved.
Resolved

Expand Down Expand Up @@ -136,6 +173,10 @@ jobs:

format-typescript:
runs-on: ubuntu-latest
timeout-minutes: 10
concurrency:
group: ${{ github.workflow }}-format-typescript-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
defaults:
run:
working-directory: crates/trusted-server-js/lib
Expand Down Expand Up @@ -167,6 +208,10 @@ jobs:

format-docs:
runs-on: ubuntu-latest
timeout-minutes: 10
concurrency:
group: ${{ github.workflow }}-format-docs-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
defaults:
run:
working-directory: docs
Expand Down Expand Up @@ -208,6 +253,10 @@ jobs:
docs-proposal-scripts:
name: docs-proposal scripts
runs-on: ubuntu-latest
timeout-minutes: 10
concurrency:
group: ${{ github.workflow }}-docs-proposal-scripts-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
name: "Integration Tests"

permissions:
Expand All @@ -10,6 +10,16 @@
types: [opened, synchronize, reopened]
workflow_dispatch:

# New pushes cancel older runs for the same pull request. Each job queues by
# workflow, job, and author, so one author runs one copy of each job at a time.
# GitHub orders jobs by when they start waiting, but does not guarantee FIFO.
# Each queue holds up to 100 pending jobs; later arrivals are canceled.
# Timeouts release queues after hung jobs. Bots and non-PR events use unique
# run IDs and do not queue together.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

env:
ORIGIN_PORT: 8888
ARTIFACTS_DIR: /tmp/integration-test-artifacts
Expand All @@ -23,6 +33,9 @@
name: prepare integration artifacts
runs-on: ubuntu-latest
timeout-minutes: 30
concurrency:
group: ${{ github.workflow }}-prepare-artifacts-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -60,6 +73,9 @@
needs: prepare-artifacts
runs-on: ubuntu-latest
timeout-minutes: 20
concurrency:
group: ${{ github.workflow }}-integration-tests-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -114,6 +130,9 @@
needs: prepare-artifacts
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: ${{ github.workflow }}-integration-tests-fastly-ec-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -158,6 +177,9 @@
# once the pinned Playwright version supports the newer image.
runs-on: ubuntu-24.04
timeout-minutes: 15
concurrency:
group: ${{ github.workflow }}-browser-tests-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/pbs-example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,24 @@ on:
permissions:
contents: read

# New pushes cancel older runs for the same pull request. Each job queues by
# workflow, job, and author, so one author runs one copy of each job at a time.
# GitHub orders jobs by when they start waiting, but does not guarantee FIFO.
# Each queue holds up to 100 pending jobs; later arrivals are canceled.
# Timeouts release queues after hung jobs. Bots and non-PR events use unique
# run IDs and do not queue together.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

jobs:
local-checks:
name: PBS Terraform and Compose checks
runs-on: ubuntu-latest
timeout-minutes: 20
concurrency:
group: ${{ github.workflow }}-local-checks-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
env:
TF_IN_AUTOMATION: 'true'
AWS_EC2_METADATA_DISABLED: 'true'
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,24 @@ on:
branches: [main]
pull_request:

# New pushes cancel older runs for the same pull request. Each job queues by
# workflow, job, and author, so one author runs one copy of each job at a time.
# GitHub orders jobs by when they start waiting, but does not guarantee FIFO.
# Each queue holds up to 100 pending jobs; later arrivals are canceled.
# Timeouts release queues after hung jobs. Bots and non-PR events use unique
# run IDs and do not queue together.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ question — In the earlier thread you planned to confirm in your throwaway repo that cancelling a superseded run also drops its jobs still waiting in a job-level queue, and to report back before merge. This PR can't show it yet: the three rework commits went up in one push, so its own history has no cancelled run. Could you post the result here? A run link is enough.


jobs:
test-rust:
name: cargo test
runs-on: ubuntu-latest
timeout-minutes: 45
concurrency:
group: ${{ github.workflow }}-test-rust-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 thinking — Following up on the bot thread with this week's data. On Oct 29 Dependabot opened 13 PRs (#1466 to #1478) within minutes, and 58 bot-authored jobs ran at once at 16:04Z across the PR workflows. Under this design one human author tops out at 24 concurrent jobs (one per job name across the five workflows), and all human authors together peaked at 59 since Oct 19. So the exempt Dependabot batch is now the largest single source of runner load. Keying bots on their login like everyone else (one dependabot[bot] queue per job) would cap it, and the per-PR cancellation would still drop their superseded rebase runs. Your call; non-blocking.

queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -71,6 +84,9 @@ jobs:
name: HTML EOF runtime (Fastly)
runs-on: ubuntu-latest
timeout-minutes: 25
concurrency:
group: ${{ github.workflow }}-test-fastly-eof-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -119,6 +135,10 @@ jobs:
test-axum:
name: cargo test (axum native)
runs-on: ubuntu-latest
timeout-minutes: 30
Comment thread
dhruv8sh marked this conversation as resolved.
Resolved
concurrency:
group: ${{ github.workflow }}-test-axum-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -192,6 +212,10 @@ jobs:
test-cloudflare:
name: cargo check (cloudflare native + wasm32-unknown-unknown)
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: ${{ github.workflow }}-test-cloudflare-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -226,6 +250,10 @@ jobs:
test-spin:
name: cargo check/build/test (spin native + wasm32-wasip1)
runs-on: ubuntu-latest
timeout-minutes: 20
concurrency:
group: ${{ github.workflow }}-test-spin-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -272,6 +300,10 @@ jobs:
test-parity:
name: cargo test (cross-adapter parity)
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: ${{ github.workflow }}-test-parity-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -306,6 +338,10 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 45
concurrency:
group: ${{ github.workflow }}-test-cli-${{ matrix.os }}-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -378,6 +414,10 @@ jobs:
name: ts dev lint domains (changed lines)
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 10
concurrency:
group: ${{ github.workflow }}-lint-domains-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
steps:
# A PR checkout is GitHub's merge commit; depth 2 also fetches its
# parents, so `HEAD^1` is the exact base commit the PR merges into.
Expand Down Expand Up @@ -412,6 +452,10 @@ jobs:
test-typescript:
name: vitest
runs-on: ubuntu-latest
timeout-minutes: 10
concurrency:
group: ${{ github.workflow }}-test-typescript-${{ github.event_name == 'pull_request' && github.event.pull_request.user.type != 'Bot' && github.event.pull_request.user.login || github.run_id }}
queue: max
defaults:
run:
working-directory: crates/trusted-server-js/lib
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,7 @@ Every PR must pass:
11. Documentation proposal scripts: `shellcheck scripts/docs-proposal/*.sh && node --test scripts/docs-proposal/hunks.test.mjs && scripts/docs-proposal/test.sh`
12. URL-host linter on lines the PR adds (`lint-domains` job, PRs only); run it locally with `cargo run --manifest-path crates/trusted-server-cli/Cargo.toml --target <host-triple> --bin ts -- dev lint domains --changed-vs origin/main`, using the PR's base branch in place of `main` when it targets another branch
13. CI Node pin guard: `node --test scripts/check-ci-node-pin.test.mjs && node scripts/check-ci-node-pin.mjs` with the `.tool-versions` Node (every CI job that compiles Rust sets up the `.tool-versions` Node before its first cargo step)
14. CI concurrency policy guard: `node --test scripts/check-ci-concurrency.test.mjs && scripts/check-ci-concurrency.sh` (requires yq v4); every PR-triggered workflow must use `test.yml`'s per-PR cancellation block, and every non-reusable job must set a timeout and a per-author `queue: max` group. Matrix jobs must also include their matrix values in the group.

CI also runs these HTML EOF checks. Run them locally only when changing EOF
finalization, publisher or HTML-processor code:
Expand Down
1 change: 1 addition & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ their documented orchestration handles a probe deliberately.
| `batch-sync.sh` | Endpoint, bearer token, EC ID, partner UID; `curl` and Python | Sends one authenticated batch-sync request. Its temporary response file is removed on exit. |
| `benchmark.sh` | A running server; `curl`, `bc`, and optionally `hey` | May install `hey` through Homebrew. `--save` writes under `benchmark-results/`; it does not stop the server. |
| `check-ci-node-pin.mjs` | Pinned Node; tests in `check-ci-node-pin.test.mjs` | Read-only. Fails when a CI job compiles Rust before an unconditional `actions/setup-node` step pinned to `.tool-versions`; `--root` checks another checkout. |
| `check-ci-concurrency.sh` | yq v4; tests in `check-ci-concurrency.test.mjs` | Read-only. Checks per-PR workflow cancellation, per-job author queues, matrix groups, and timeouts; `--root` checks another checkout and `--yq` selects yq. |
| `profile.sh` | Fastly CLI, Rust WASM target, `curl`; endpoint and request options | Builds and starts the Fastly app, stops the owned process, and retains a profile under `benchmark-results/profiles/`. `--open` launches the local viewer. |
| `generate-integration-viceroy-configs.sh` | Rust toolchain; optional origin port and artifact directory | Builds the native generator and writes Viceroy config under `target/integration-test-artifacts/`; generated files persist. |
| `integration-tests.sh` | Docker, Viceroy, Rust WASM target, pinned Node | Builds WASM/native artifacts and two Docker images, generates Viceroy config, and runs native integration tests serially. Build products and images persist. |
Expand Down
Loading
Loading