From 56e173e87eeb2dfa5e4c9aae9b4d125c01c78fc8 Mon Sep 17 00:00:00 2001 From: Alice Frosi Date: Tue, 27 Oct 2026 09:50:22 +0000 Subject: [PATCH] e2e: seed node image with skopeo to preserve its digest GitHub moved the ubuntu-latest runner from Ubuntu 24.04 to 26.04, which upgraded podman from 4.9.3 to 5.7.0. Since then most of the bink e2e tests fail on every Kubernetes version with: Value for field 'Booted' failed to satisfy matcher. Value for field 'ImageDigest' failed to satisfy matcher. Expected "...a256:9566a5..." to equal "...a256:0136fd..." seed-node-image pulls the bootc image that the node disk image was installed from (by digest) and re-pushes it to the local bink registry with `podman push`. The tests take the "original" digest from the local registry (E2E_NODE_IMAGE_DIGEST) and expect the node's booted digest to match it. podman 4 pushed the OCI manifest unchanged, so both digests were sha256:9566a5.... podman 5 rewrites the manifest on push, so the local registry now holds sha256:0136fd... while the VM is still booted from sha256:9566a5.... Every test that checks the booted digest against the original image now times out. Tests that only check the update images still pass, because those images are built and pushed locally and their digests are read back from the registry. Copy the image directly from the source registry to the local registry with `skopeo copy --preserve-digests` so the manifest is transferred unchanged, independent of the podman version on the runner. The local podman tag is kept because build-update-image builds FROM localhost:5000/node:latest. Assisted-by: AI Signed-off-by: Alice Frosi (cherry picked from commit dd88a5760f4ad105bb877850a43d5712f4c927bd) --- Makefile | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index aafcdeef..e222078c 100644 --- a/Makefile +++ b/Makefile @@ -228,6 +228,8 @@ undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/. # Note the :latest tag here: this makes the pull policy be Always. IMG_BINK ?= registry.cluster.local:5000/bootc-operator-e2e:latest +# Copy with skopeo rather than `podman push`: the node disk image booted the +# source digest, and podman 5 rewrites the manifest on push, which changes it. .PHONY: seed-node-image seed-node-image: ## Pull the bootc node image by digest and push to the bink registry. bink registry start @@ -235,8 +237,10 @@ seed-node-image: ## Pull the bootc node image by digest and push to the bink reg bootc_img=$$(podman inspect --format '{{index .Config.Labels "bink.bootc-image"}}' $(BINK_NODE_DISK_IMAGE)) && \ bootc_digest=$$(podman inspect --format '{{index .Config.Labels "bink.bootc-image-digest"}}' $(BINK_NODE_DISK_IMAGE)) && \ podman pull "$$bootc_img@$$bootc_digest" && \ - podman tag "$$bootc_img@$$bootc_digest" localhost:5000/node:latest - podman push --tls-verify=false localhost:5000/node:latest + podman tag "$$bootc_img@$$bootc_digest" localhost:5000/node:latest && \ + bootc_repo=$${bootc_img%:*} && \ + skopeo copy --preserve-digests --dest-tls-verify=false \ + "docker://$$bootc_repo@$$bootc_digest" docker://localhost:5000/node:latest .PHONY: start-bink start-bink: seed-node-image ## Start a bink cluster (idempotent).