Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions .github/workflows/publish-tool-images.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Publish tool images

# Builds the reference images in tool-images/ (the tool-cache and system-library capability images, see the ARC role README's Capabilities section, and the build runner image, the stock runner image with a C toolchain) for linux/amd64 and linux/arm64 and pushes them to this repository owner's GitHub Container Registry namespace, as ghcr.io/<owner>/github-runner-toolcache-<name>, ghcr.io/<owner>/github-runner-sysroot-<name> and ghcr.io/<owner>/github-runner-runner-<name>, each tagged with the version its Dockerfile pins (its ARG <TOOL>_VERSION default; for the build runner image, the runner version it extends). A version tag that already exists is left alone, so a published version never changes under anyone pinning it and a run publishes only the images whose pinned version is new. The job summary lists each pushed image by digest, the reference to pin in a capability or as a runner image.
# Builds the reference images in tool-images/ (the tool-cache and system-library capability images, see the ARC role README's Capabilities section, and the build runner image, the stock runner image with a C toolchain) for linux/amd64 and linux/arm64 and pushes them to this repository owner's GitHub Container Registry namespace, as ghcr.io/<owner>/github-runner-toolcache-<name>, ghcr.io/<owner>/github-runner-sysroot-<name> and ghcr.io/<owner>/github-runner-runner-<name>, each tagged with the version it builds: the one its matrix entry names in `version`, built through the Dockerfile's ARG <TOOL>_VERSION, or, for an entry without one, the version that ARG defaults to (for the build runner image, the runner version it extends). One directory can therefore publish several versions of a tool, each its own tag of the same repository. A version tag that already exists is left alone, so a published version never changes under anyone pinning it and a run publishes only the images whose pinned version is new. The job summary lists each pushed image by digest, the reference to pin in a capability or as a runner image.
#
# Runs only when started by hand or by pushing a tag named tool-images-<anything>; the release workflow does not build these images. Each architecture is built natively, on GitHub's amd64 and arm64 hosted runners, and pushed by digest, and a second job joins the two into one multi-architecture tag (https://docs.docker.com/build/ci/github-actions/multi-platform/), because compiling the system libraries under emulation is far slower than natively.

Expand All @@ -19,7 +19,7 @@ permissions:

jobs:
build:
name: Build ${{ matrix.image.kind }}-${{ matrix.image.name }} (${{ matrix.platform.arch }})
name: Build ${{ matrix.image.kind }}-${{ matrix.image.name }} ${{ matrix.image.version }} (${{ matrix.platform.arch }})
runs-on: ${{ matrix.platform.runner }}
timeout-minutes: 90
permissions:
Expand All @@ -30,6 +30,7 @@ jobs:
matrix:
image:
- {name: node, kind: toolcache}
- {name: node, kind: toolcache, version: 26.12.1}
- {name: terraform, kind: toolcache}
- {name: tflint, kind: toolcache}
- {name: awscli, kind: toolcache}
Expand All @@ -51,15 +52,21 @@ jobs:
OWNER: ${{ github.repository_owner }}
KIND: ${{ matrix.image.kind }}
NAME: ${{ matrix.image.name }}
PINNED_VERSION: ${{ matrix.image.version }}
run: |
set -euo pipefail
version="$(sed -n 's/^ARG [A-Z_]*_VERSION=//p' "tool-images/${NAME}/Dockerfile")"
if [ -z "$version" ] || [ "$(wc -l <<<"$version")" != 1 ]; then
pins="$(sed -n 's/^ARG \([A-Z_]*_VERSION\)=\(.*\)/\1 \2/p' "tool-images/${NAME}/Dockerfile")"
if [ -z "$pins" ] || [ "$(wc -l <<<"$pins")" != 1 ]; then
echo "tool-images/${NAME}/Dockerfile must pin exactly one ARG <TOOL>_VERSION" >&2
exit 1
fi
echo "repository=ghcr.io/${OWNER,,}/github-runner-${KIND}-${NAME}" >>"$GITHUB_OUTPUT"
echo "version=${version}" >>"$GITHUB_OUTPUT"
argument="${pins% *}"
version="${PINNED_VERSION:-${pins#* }}"
{
echo "repository=ghcr.io/${OWNER,,}/github-runner-${KIND}-${NAME}"
echo "version=${version}"
echo "argument=${argument}"
} >>"$GITHUB_OUTPUT"

- uses: docker/login-action@v4
with:
Expand Down Expand Up @@ -88,6 +95,8 @@ jobs:
with:
context: tool-images/${{ matrix.image.name }}
platforms: linux/${{ matrix.platform.arch }}
build-args: |
${{ steps.image.outputs.argument }}=${{ steps.image.outputs.version }}
labels: |
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
Expand All @@ -106,13 +115,13 @@ jobs:
- uses: actions/upload-artifact@v7
if: steps.published.outputs.exists == 'false'
with:
name: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-${{ matrix.platform.arch }}
name: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-${{ steps.image.outputs.version }}-${{ matrix.platform.arch }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1

merge:
name: Publish ${{ matrix.image.kind }}-${{ matrix.image.name }}
name: Publish ${{ matrix.image.kind }}-${{ matrix.image.name }} ${{ matrix.image.version }}
needs: [build]
# Each image is joined on its own, so one image's failed build does not hold back the others; the join itself refuses an image missing an architecture.
if: ${{ !cancelled() }}
Expand All @@ -126,6 +135,7 @@ jobs:
matrix:
image:
- {name: node, kind: toolcache}
- {name: node, kind: toolcache, version: 26.12.1}
- {name: terraform, kind: toolcache}
- {name: tflint, kind: toolcache}
- {name: awscli, kind: toolcache}
Expand All @@ -144,16 +154,17 @@ jobs:
OWNER: ${{ github.repository_owner }}
KIND: ${{ matrix.image.kind }}
NAME: ${{ matrix.image.name }}
PINNED_VERSION: ${{ matrix.image.version }}
run: |
set -euo pipefail
version="$(sed -n 's/^ARG [A-Z_]*_VERSION=//p' "tool-images/${NAME}/Dockerfile")"
default="$(sed -n 's/^ARG [A-Z_]*_VERSION=//p' "tool-images/${NAME}/Dockerfile")"
echo "repository=ghcr.io/${OWNER,,}/github-runner-${KIND}-${NAME}" >>"$GITHUB_OUTPUT"
echo "version=${version}" >>"$GITHUB_OUTPUT"
echo "version=${PINNED_VERSION:-${default}}" >>"$GITHUB_OUTPUT"

- uses: actions/download-artifact@v8
with:
path: ${{ runner.temp }}/digests
pattern: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-*
pattern: digests-${{ matrix.image.kind }}-${{ matrix.image.name }}-${{ steps.image.outputs.version }}-*
merge-multiple: true

- uses: docker/login-action@v4
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ A release:

The `[skip ci]` in the release commit message is deliberate: the image build already happens in this same run using the exact version semantic-release just decided, so there is nothing useful for a second, separately-triggered run of this workflow (or of `ci.yml`) to do against a commit that only changed a version stamp and a changelog - `ci.yml`'s own lint jobs already ran against every commit before it reached `main`.

The reference capability images in `tool-images/` are not part of a release. `.github/workflows/publish-tool-images.yml` builds each one natively on amd64 and arm64 runners and pushes it to `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` or `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version its Dockerfile pins, when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a `tool-images-<anything>` tag. It leaves a version tag that already exists alone, and its run summary gives each pushed image's `<image>:<version>@sha256:<digest>` reference to pin in a capability or as a runner image. A new GitHub Container Registry package starts private; make it public in its package settings so pods can pull it without a credential.
The reference capability images in `tool-images/` are not part of a release. `.github/workflows/publish-tool-images.yml` builds each one natively on amd64 and arm64 runners and pushes it to `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` or `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version it builds (a matrix entry's `version`, else the default of its Dockerfile's `ARG <TOOL>_VERSION`; a tool kept at several versions has an entry for each), when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a `tool-images-<anything>` tag. It leaves a version tag that already exists alone, and its run summary gives each pushed image's `<image>:<version>@sha256:<digest>` reference to pin in a capability or as a runner image. A new GitHub Container Registry package starts private; make it public in its package settings so pods can pull it without a credential.

**What the release push needs:** a deploy key with write access on `ExaDev/github-runner`, whose private key is the `RELEASE_DEPLOY_KEY` repository secret `release.yml` checks out with. If `main` gains a ruleset, add deploy keys as its bypass actor so the release commit can land. Publishing to Ansible Galaxy additionally needs a `GALAXY_API_KEY` repository secret (an API key from `https://galaxy.ansible.com/me/preferences`); without it, the publish step logs that it's skipping and the release still completes.

Expand Down
4 changes: 3 additions & 1 deletion roles/github_runner_arc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,8 @@ docker buildx build --platform linux/amd64,linux/arm64 --build-arg NODE_VERSION=
-t ghcr.io/example/toolcache-node:24.21.0 --push tool-images/node
```

One Dockerfile builds every version of a tool: the version is its `ARG <TOOL>_VERSION`, and the image for another version is the same command with that value (the Node.js image checks each archive against the release's `SHASUMS256.txt` and installs nothing, so it runs no package manager). A fleet that needs two Node.js majors names the two tags as two capabilities (`node-24`, `node-26`): their tool caches are merged under `<tool cache>/node/<version>/`, and `actions/setup-node` takes whichever cached version satisfies its `node-version`, so `>=26` finds the second and `24` the first. Name `path` on one of them only if a tool no setup action looks up should get that version on `PATH`.

A capability image of your own takes a few lines. This one packages a static binary, `example-cli`, built beside the Dockerfile for each architecture, for use through `path`:

```dockerfile
Expand Down Expand Up @@ -319,7 +321,7 @@ Then prove it the way `tests/system_libraries/run.sh` does before relying on it:

### Publishing the reference images

`.github/workflows/publish-tool-images.yml` builds every reference image in `tool-images/` natively for `linux/amd64` and `linux/arm64` and pushes it to the repository owner's GitHub Container Registry namespace, as `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` and, for the build runner image, `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version its Dockerfile pins (for the build runner image, the stock runner version it extends, so a new runner release is a new tag). It runs only when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a tag named `tool-images-<anything>`; the release workflow does not build these images. A version tag that already exists is left alone, so a published version never changes under anyone using it, and a run publishes only the images whose pinned version is new; to publish a new version, change the Dockerfile's version (and, where it pins one, its checksum) and run it again. The run's summary prints each pushed image as `<image>:<version>@sha256:<digest>`, the reference to put in a capability. A newly created GitHub Container Registry package is private until its visibility is changed to public in the package's settings, which pods without a pull credential need.
`.github/workflows/publish-tool-images.yml` builds every reference image in `tool-images/` natively for `linux/amd64` and `linux/arm64` and pushes it to the repository owner's GitHub Container Registry namespace, as `ghcr.io/<owner>/github-runner-toolcache-<name>`, `ghcr.io/<owner>/github-runner-sysroot-<name>` and, for the build runner image, `ghcr.io/<owner>/github-runner-runner-build`, tagged with the version it builds: the one its matrix entry names in `version`, or, for an entry without one, the default of the Dockerfile's `ARG <TOOL>_VERSION` (for the build runner image, the stock runner version it extends, so a new runner release is a new tag). It runs only when started by hand (`gh workflow run publish-tool-images.yml`) or by pushing a tag named `tool-images-<anything>`; the release workflow does not build these images. A version tag that already exists is left alone, so a published version never changes under anyone using it, and a run publishes only the images whose pinned version is new; to publish a new version, change the Dockerfile's version (and, where it pins one, its checksum) and run it again. A tool kept at more than one version has an entry per version in the workflow's matrix, `{name: node, kind: toolcache, version: 26.12.1}` beside the plain `{name: node, kind: toolcache}` entry that builds the Dockerfile's default, listed in both the `build` and `merge` matrices (a unit test fails if they differ); a bump of such a version is a new `version` value there, which publishes a new tag of the same repository and leaves the old one in place for the fleets still pinning it. The Node.js image takes the new version's archive checksums from nodejs.org's `SHASUMS256.txt` at build time, so a bump needs no checksum edit; compare them with the release's signed copy (`SHASUMS256.txt.asc`) when you choose the version. The run's summary prints each pushed image as `<image>:<version>@sha256:<digest>`, the reference to put in a capability. A newly created GitHub Container Registry package is private until its visibility is changed to public in the package's settings, which pods without a pull credential need.

### The job-started hook

Expand Down
Loading
Loading