diff --git a/.github/ISSUE_TEMPLATE/release.md b/.github/ISSUE_TEMPLATE/release.md new file mode 100644 index 0000000..74a82df --- /dev/null +++ b/.github/ISSUE_TEMPLATE/release.md @@ -0,0 +1,72 @@ +--- +name: Release +about: One monthly release of arcadedb-jvm and arcadedb (native), step by step +title: "Release " +labels: "" +--- + +Open this issue once per release, when upstream ArcadeDB has released an engine version. Replace the placeholders everywhere on this page before you start: + +- ``: the tag to push, for example the engine release `A.B.C`. A Python-only fix of a released engine is `A.B.C.postN`. +- ``: the previous release of the same package on PyPI (the JVM and the native package can differ), or `-` for the first release of a package. +- ``: the first dev version of the next base, `A.B.C.dev0` (see step 7). + +Every step says who does it: script (run the command, read the result), CI (a workflow does it, you wait), or owner (a decision that is not automated). The procedure and the reasons are on the docs page `development/release.adoc`, section "Release day". + +## Before the tag + +- [ ] 1. script: upstream `` is out on both channels (GitHub release, Maven Central, Docker Hub image), the version rule holds for both packages, the CI of `main` is green, and the release workflows derive the right version and engine. Zero FAILs means ready. A FAIL on Maven Central or Docker Hub can mean the other channel is still catching up: run it again later. + + ```bash + uv run --no-project python -I scripts/release_preflight.py + ``` + +- [ ] 2. owner: for a final or post release, the native readiness bar is met and the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET` is `true` (the preflight line `(g)` shows it, or SKIPs when the login cannot read variables). +- [ ] 3. owner: the go. Nothing is published before this box is ticked. + +## The tag (the only publishing step) + +- [ ] 4. owner: push the tag. It is the bare version, with no `v`, on the tip of `main`. One tag starts `release-jvm.yml` and `release-native.yml`. + + ```bash + git fetch origin main --no-tags && git tag origin/main && git push origin + ``` + +- [ ] 5. CI: both release workflows finish and publish. Note the two run ids (the native build takes the longest). To retry a failed upload, re-run only the publish job, never the whole workflow. + + ```bash + gh run list -R ArcadeData/arcadedb-python --branch --limit 6 + gh run watch -R ArcadeData/arcadedb-python + ``` + +## After the tag + +- [ ] 6. script: verify both packages on PyPI. Run them under bash. The scripts read from GitHub and PyPI only, and download about 0.6 GB: point `WORK` at a disk, not a tmpfs. The exit code is the number of FAILs. + + ```bash + scripts/after_tag_verify.sh + PACKAGE=native scripts/after_tag_verify.sh + ``` + +- [ ] 7. script, then CI: the dev-engine bump PR. Open it from a worktree on a branch off `origin/main`. Pick `` from the version of upstream `main`: the command prints the next base. If it prints ``, upstream has not moved on yet: wait, because a dev version of a released base is refused. + + ```bash + curl -fsS https://raw.githubusercontent.com/ArcadeData/arcadedb/main/pom.xml | sed -n '/arcadedb-parent<\/artifactId>/,/<\/version>/ s/.*\(.*\)-SNAPSHOT<\/version>.*/\1/p' + # is that base followed by .dev0 + uv run --no-project python scripts/bump_engine.py --engine --dev-version --check + uv run --no-project python scripts/bump_engine.py --engine --dev-version + ``` + +- [ ] 8. script: refresh the notices the new engine needs. The first `Test native` run on the bump PR stops at the notices drift check on purpose and uploads the generated file. Download it, commit it, push, and wait for the run to go green. + + ```bash + gh run list -R ArcadeData/arcadedb-python --workflow test-native.yml --branch --limit 1 + gh run download -R ArcadeData/arcadedb-python -n native-build-record-linux-x86_64 -D + cp /THIRD_PARTY_NOTICES.generated.md native/wheel/THIRD_PARTY_NOTICES.md + uv run --no-project python scripts/bump_engine.py --engine --dev-version --check + ``` + + If the engine NOTICE changed, also refresh the kept copy with `git -C show :NOTICE > native/wheel/ARCADEDB_ENGINE_NOTICE`. + +- [ ] 9. CI, then owner: merge the bump PR when every check is green, and delete the branch and the worktree. Then remove the tests and skip-list lines that waited for the new engine: `git grep -n -i "native build links\|native image links"`. +- [ ] 10. owner: close this issue with the two run ids and the two PyPI links. diff --git a/CLAUDE.md b/CLAUDE.md index 67efcda..332cf3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,6 +73,6 @@ Each package has one test and one release workflow: `test-jvm.yml` and `release- `.github/workflows/test-jvm.yml` builds ONE wheel per platform (linux/amd64, linux/arm64, darwin/arm64, windows/amd64), tagged `py3-none-` by `setup.py` (`bdist_wheel.get_tag`; `scripts/verify_wheel_tag.py` checks it), and tests that wheel on Python 3.11–3.14; also runs nightly against `arcadedata/arcadedb:latest`. `requires-python` is `>=3.11,<3.15` because the tag no longer keeps pip off 3.15 and JPype has no 3.15 wheel; drop the cap with it. The org shares a runner cap with upstream, so the unit tests run 10 legs on every event (linux/amd64 on all four Pythons, the other three platforms on 3.11 and 3.14, the shape of test-native.yml), and the examples run all four platforms on 3.14 on a pull request (4 legs, 28 checks in all) and 16 legs on push to main, the nightly run, dispatch and the release (40 checks). `scripts/select_matrix.py` holds both lists (`PULL_REQUEST_LEGS`, keyed `jvm` and `examples`), the first job of each workflow runs it, and `tests/test_select_matrix.py` pins the full matrix; docs page `development/ci.adoc`. Keep the job names `Test arcadedb (, Python )` unchanged. Both workflows accept an `arcadedb-tag` input to pin the image. `test-python-examples.yml` runs `examples/`. A dependency-floor audit locks the dependencies and extras on a throwaway uv project with `--resolution lowest-direct` for each Python classifier and runs `uvx pip-audit` on the export, so raise lower bounds when a floor has an advisory. No tool is installed with pip anywhere in CI or the scripts: one-off tools run through `uvx tool==X`, the wheel is built with `uvx --from "build[uv]==1.6.1" pyproject-build --installer uv`, and the uv install URL is pinned (`0.12.0`). -`.github/workflows/release-jvm.yml` publishes `arcadedb-jvm` on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/after_tag_verify.sh` checks the finished release. The rule is stated once, on the docs page `development/release.adoc`. +`.github/workflows/release-jvm.yml` publishes `arcadedb-jvm` on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/release_preflight.py ` checks everything before the tag (read-only); `scripts/after_tag_verify.sh` (`PACKAGE=native` for `arcadedb`) checks the finished release; `scripts/bump_engine.py` moves the dev-engine pins after a release. The monthly steps are the "Release day" section of `development/release.adoc` and the issue template `.github/ISSUE_TEMPLATE/release.md`. The rule is stated once, on the docs page `development/release.adoc`. `test-native.yml` builds the GraalVM native image and runs the parity suite against the installed wheel (selective: scripts/select_native_matrix.py, 5 to 30 minutes per platform, do not add runs lightly). `.github/workflows/release-native.yml` publishes `arcadedb` on the same tag: it calls `test-native.yml` with `release: true` (4 platforms x Python 3.11 to 3.14), then the readiness gate (the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET`, for a final or post version), then one publish job through the environment `pypi`. diff --git a/README.md b/README.md index e9c89c1..a4186b9 100644 --- a/README.md +++ b/README.md @@ -98,7 +98,9 @@ Releases from this repository start with 26.11.1. The 26.10.x wheels were built The `arcadedb` (native) wheel is built from `native/` with GraalVM Community Edition; [`CLAUDE.md`](https://github.com/ArcadeData/arcadedb-python/blob/main/CLAUDE.md) has the commands. A native image build takes about 5 to 30 minutes per platform (macOS is the slowest). -The JVM build runs in Docker on Linux. macOS and Windows need a JDK 25 or later. The jars come from the `arcadedata/arcadedb` Docker image: `latest` by default, or the tag in `ARCADEDB_TAG` (for example `ARCADEDB_TAG=26.10.2 ./scripts/build.sh`). The package version is read from the engine jar in that image, so a `26.11.1-SNAPSHOT` image builds `26.11.1.devN`. +The JVM build runs in Docker on Linux. macOS and Windows need a JDK 25 or later. The jars come from the `arcadedata/arcadedb` Docker image: `latest` by default, or the tag in `ARCADEDB_TAG` (for example `ARCADEDB_TAG=26.10.2 ./scripts/build.sh`). The package version is read from the engine jar in that image, so an `A.B.C-SNAPSHOT` image builds `A.B.C.devN`. + +The native build does not use the image. It resolves `arcadedb-engine` and `arcadedb-server` from Maven Central at `arcadedb.version` in `native/pom.xml` (release versions only, because a snapshot is not on Maven Central) and compiles them with GraalVM Community Edition. Neither build compiles the engine from source: the engine code is upstream's, unmodified. Tests run against the built wheel, not the source tree: install the wheel with its `test` extra into a virtual environment, then run `pytest`. [`CLAUDE.md`](https://github.com/ArcadeData/arcadedb-python/blob/main/CLAUDE.md) has the exact commands, the build options and the lint setup. diff --git a/docs/modules/ROOT/pages/development/ci.adoc b/docs/modules/ROOT/pages/development/ci.adoc index 4653a95..1610a65 100644 --- a/docs/modules/ROOT/pages/development/ci.adoc +++ b/docs/modules/ROOT/pages/development/ci.adoc @@ -54,6 +54,15 @@ The names of the test jobs do not change: `Test arcadedb (, Python --git-remote origin --pypi arcadedb-jvm ---- It exits 0 when the version is the correct next one, 1 when it is not, and 2 when a source cannot be read. Exit 2 is a failure, not a pass: the check fails closed. @@ -48,20 +48,23 @@ The tests of the script are in `tests/test_check_version.py`. == After the tag push -`scripts/after_tag_verify.sh` checks the finished `arcadedb-jvm` release (the run of `release-jvm.yml`). It only reads from GitHub and PyPI: +`scripts/after_tag_verify.sh` checks a finished release: `arcadedb-jvm` (the run of `release-jvm.yml`) by default, and the native `arcadedb` (the run of `release-native.yml`) with `PACKAGE=native`. It only reads from GitHub and PyPI: [source,bash] ---- -scripts/after_tag_verify.sh 26.11.1 26.10.1 +scripts/after_tag_verify.sh +PACKAGE=native scripts/after_tag_verify.sh ---- -The second argument is the previous release. Use `-` for the first release of the package. The script checks: +The second argument is the previous release of that package. Use `-` for the first release of the package. The script checks, for the JVM package: * The workflow run, and the conclusion of the publish job and of its upload step. * The files on PyPI: one `py3-none` wheel for each platform (the same file for every Python version), each under 100 MB, none yanked. * The sha256 of every file on PyPI against the wheels the workflow checked. * A download of the wheel of each of the four platforms from PyPI (picked from the PyPI JSON, fetched with `curl`, sha256 compared with the one PyPI lists), and a fresh `uv run --with` install on Python 3.12 and 3.14 that opens a database and runs one SQL and one openCypher query. +For the native package the steps are the same with three differences: the checked wheels are the artifact `native-release-dist`; `windows-x64` is required for a final release and optional for a `devN` release (as `release-native.yml` publishes it); and the smoke run asserts the native backend and that `ENGINE_VERSION` is the engine the tag is named after, because there is no jar to check. + It prints `PASS` or `FAIL` for each step, and its exit code is the number of failures. [[_native_release]] @@ -93,8 +96,98 @@ The version rule above holds for both packages. One tag starts both release work `arcadedb` has one trusted publisher on PyPI (`release-native.yml` with the environment `pypi`) for dev, final and post releases, and one publish job in the workflow. `arcadedb-jvm` has one trusted publisher (`release-jvm.yml` with the environment `pypi-jvm`). PyPI matches the workflow file name, so renaming either file means changing the publisher on PyPI first. There is no separate pre-release environment; the readiness gate below is what keeps a final release back. -* A native pre-release (`26.11.1.devN`) publishes when the builds on `linux-x86_64`, `linux-aarch64` and `macos-arm64` pass. +* A native pre-release (`A.B.C.devN`) publishes when the builds on `linux-x86_64`, `linux-aarch64` and `macos-arm64` pass. * A FINAL native release (`A.B.C` or `A.B.C.postN`) needs two more things: ** The repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET` is `true`. A maintainer sets it after checking the readiness bar: the correctness fixes are merged; the full suite is green on every platform, Windows included, and on Python 3.11 to 3.14; the fault and concurrency probes pass; and the documented API raises no `NotOnNativeYet`. ** The parity run of every platform and every Python version passes with zero `NotOnNativeYet` skips. * Speed parity with the JVM build is not part of the bar. + +[[_release_day]] +== Release day + +A release happens about once a month, after upstream ArcadeDB releases an engine version. Everything below takes the version as an argument; nothing is typed into a script. `` is the tag to push (the engine release `A.B.C`, or `A.B.C.postN` for a Python-only fix of a released engine), `` is the previous release of the package on PyPI (`-` for the first one), and `` is the first dev version of the next base. The checklist with every command is the issue template `.github/ISSUE_TEMPLATE/release.md` (https://github.com/ArcadeData/arcadedb-python/issues/new?template=release.md[open it once per release]); open it first and tick the boxes in order. + +The two packages read the engine from two different channels of the same upstream release: + +* The JVM wheels take the engine jars from the Docker Hub image `arcadedata/arcadedb:`. +* The native build resolves `arcadedb-engine` and `arcadedb-server` from Maven Central at that version and compiles them with GraalVM CE `native-image`. + +Both channels must carry ``. They are published by different upstream steps and can appear hours apart, so a release is ready only when the preflight passes both. + +Each step is a script, CI, or the owner's decision. The owner's tag push is the only step that publishes anything. + +[cols="1,4,2"] +|=== +| Step | What | Who + +| 1 +| Preflight. Checks that the upstream GitHub release exists (not draft), that Maven Central has `arcadedb-engine` and `arcadedb-server`, that Docker Hub has the image for `linux/amd64` and `linux/arm64`, that `scripts/check_version.py` accepts `` for both packages against the git tags and PyPI, that the latest conclusive runs of `Test native`, `Test JVM` and `Test Python Examples` on `main` are green on every job, that the release workflows derive `` and the engine from the tag (the real step code of the workflows runs with `TAG=`), and that the native readiness variable is set for a final release. It only reads, prints `PASS`, `FAIL` or `SKIP` with the evidence, fails closed on network errors, and exits with the number of FAILs. +| script: `scripts/release_preflight.py` + +| 2 +| The go. Zero FAILs, the readiness bar met for a final or post release (the `SKIP` on the variable means the login cannot read it, so check it), and a decision to publish. +| owner + +| 3 +| Push the tag, the bare version on the tip of `main`. One tag starts both release workflows. +| owner (the only publishing step) + +| 4 +| `release-jvm.yml` and `release-native.yml` validate the tag, build and test the wheels, check them, and publish with trusted publishing. A failed upload is retried by re-running only the publish job. +| CI + +| 5 +| After-tag verification of both packages: the workflow run and its publish step, the file set and sizes on PyPI, the sha256 of every file against the wheels the workflow checked, and a fresh `uv run --with` install on Python 3.12 and 3.14 that opens a database and runs one SQL and one openCypher query. +| script: `scripts/after_tag_verify.sh` and `PACKAGE=native scripts/after_tag_verify.sh` + +| 6 +| The dev-engine bump PR. `scripts/bump_engine.py` moves the pins that dev, PR and nightly builds link: `DEV_ENGINE`, `DEFAULT_VERSION` and the default of the manual-run input `version` in `test-native.yml`, `arcadedb.version` in `native/pom.xml`, the default in `native/scripts/build-image.sh`, and the default of `--version` in `native/wheel/build_wheel.py`. `--check` reports drift and writes nothing; it refuses a `` that `check_version.py` would refuse. +| script: `scripts/bump_engine.py` + +| 7 +| The notices of the new engine (below), then the CI of the bump PR, then the merge on green. +| script, CI, then merge +|=== + +=== Commands + +[source,bash] +---- +# 1. preflight +uv run --no-project python -I scripts/release_preflight.py + +# 3. the tag (owner) +git fetch origin main --no-tags && git tag origin/main && git push origin + +# 5. after the workflows finished (run ids: gh run list -R ArcadeData/arcadedb-python --branch ) +scripts/after_tag_verify.sh +PACKAGE=native scripts/after_tag_verify.sh + +# 6. the bump +uv run --no-project python scripts/bump_engine.py --engine --dev-version --check +uv run --no-project python scripts/bump_engine.py --engine --dev-version +---- + +=== Choosing the next dev version + +A dev version of a released base is refused, so after `` is out the dev line moves to the next base. The next base is the version of upstream `main`, which carries `-SNAPSHOT`: + +[source,bash] +---- +curl -fsS https://raw.githubusercontent.com/ArcadeData/arcadedb/main/pom.xml | sed -n '/arcadedb-parent<\/artifactId>/,/<\/version>/ s/.*\(.*\)-SNAPSHOT<\/version>.*/\1/p' +---- + +`` is that base followed by `.dev0`. When the command prints `` itself, upstream has not moved `main` on yet: wait, and do not guess a base. The JVM dev wheels need no pin: they take the jars of the image tag `latest`, which follows upstream. + +=== The notices of the new engine + +`native/wheel/THIRD_PARTY_NOTICES.md` names the engine and is generated from the build log of the native image, so `build_wheel.py` fails on any difference. `bump_engine.py` reports it as `TODO` and does not edit it, and its `--check` stays at exit 1 until the file names the new engine. To regenerate: + +. Push the bump PR. The first `Test native` run stops at the notices drift check on purpose, and its `native-build-record-linux-x86_64` artifact holds `THIRD_PARTY_NOTICES.generated.md`, the output of `gen_notices.py` for the new engine (the text is the same on every platform). +. Download the artifact with `gh run download -R ArcadeData/arcadedb-python -n native-build-record-linux-x86_64 -D `, copy `THIRD_PARTY_NOTICES.generated.md` over `native/wheel/THIRD_PARTY_NOTICES.md`, and commit it to the bump branch. +. When the engine `NOTICE` changed, refresh the kept copy: `git -C show :NOTICE > native/wheel/ARCADEDB_ENGINE_NOTICE`. When the next run still fails on `NOTICE`, run the `gen_notices.py ... --notice` command that the failure message prints. +. Check with `uv run --no-project python scripts/bump_engine.py --engine --dev-version --check`, which must exit 0, and wait for the CI of the PR to turn green. + +Some tests and skip lists wait for the engine the native image links, and say so in a comment (for example the `29_hybrid_memory_retrieval.py` line of `scripts/native_examples_skip.txt`). After the bump merges, find them with `git grep -n -i "native build links\|native image links"` and remove the ones the new engine settles. + +The tests of the two scripts are `tests/test_release_preflight.py` and `tests/test_bump_engine.py`. diff --git a/scripts/after_tag_verify.sh b/scripts/after_tag_verify.sh index c211a43..41dd4dc 100755 --- a/scripts/after_tag_verify.sh +++ b/scripts/after_tag_verify.sh @@ -1,9 +1,16 @@ #!/bin/bash # After the tag push: verify the release end to end. Read-only against GitHub and PyPI (no writes). # Ported from humemai/arcadedb-embedded-python#479 and adapted to .github/workflows/release-jvm.yml (the arcadedb-jvm release; the artifact it reads is release-dist). +# PACKAGE=native checks the native `arcadedb` release instead (release-native.yml, artifact native-release-dist). # -# usage: after_tag_verify.sh [] [ ] -# example: after_tag_verify.sh 26.11.1 26.10.1 +# usage: [PACKAGE=jvm|native] after_tag_verify.sh [] [ ] +# example: after_tag_verify.sh +# example: PACKAGE=native after_tag_verify.sh +# +# PACKAGE=native differs in four things: the distribution is `arcadedb`; the checked wheels are the artifact native-release-dist; +# the platform windows-x64 is required for a final release (A.B.C, A.B.C.postN) and optional for A.B.C.devN, as release-native.yml +# publishes it; and the smoke run asserts the native backend and that ENGINE_VERSION is the engine the tag is named after (no jar +# to check, so the last two arguments are not used). # # is the release whose PyPI file set this one must match in shape. Pass - when # there is none (the first release of the distribution): the file set is then checked against @@ -18,7 +25,24 @@ RUN="${3:-}" ENG_SHA="${4:-}" ENG_BUILD="${5:-}" R=ArcadeData/arcadedb-python -DIST=arcadedb-jvm +PACKAGE="${PACKAGE:-jvm}" +case "$PACKAGE" in + jvm) + DIST=arcadedb-jvm + ARTIFACT=release-dist + WIN_OPTIONAL=0 + ;; + native) + DIST=arcadedb + ARTIFACT=native-release-dist + # a devN native release publishes Windows only when it passed; a final or post release requires it + case "$V" in *.dev*) WIN_OPTIONAL=1 ;; *) WIN_OPTIONAL=0 ;; esac + ;; + *) + printf 'PACKAGE must be jvm or native, got %s\n' "$PACKAGE" >&2 + exit 2 + ;; +esac W="${WORK:-$(mktemp -d "${TMPDIR:-$HOME}/after-tag-XXXXXX")}" FAILS=0 pass() { printf 'PASS %s\n' "$*"; } @@ -51,9 +75,10 @@ if [ "$PREV" = "-" ]; then else curl -fsS "https://pypi.org/pypi/$DIST/$PREV/json" -o "$W/pypi-prev.json" || fail "cannot read PyPI $PREV" fi -python3 - "$W/pypi-new.json" "$W/pypi-prev.json" "$V" "$PREV" << 'EOF' || FAILS=$((FAILS + 1)) +python3 - "$W/pypi-new.json" "$W/pypi-prev.json" "$V" "$PREV" "$WIN_OPTIONAL" << 'EOF' || FAILS=$((FAILS + 1)) import json,re,sys new,prev,v,pv=json.load(open(sys.argv[1])),json.load(open(sys.argv[2])),sys.argv[3],sys.argv[4] +win_optional=sys.argv[5]=="1" # native devN: windows-x64 is published only when it passed # Releases up to 26.11.1.dev0 had one cpXY-cpXY wheel for each Python and platform; since then the wheel is py3-none-, # one for each platform. Read the previous file set in the new shape, so the Pythons a release drops or adds change no file. want={re.sub(r"-cp3\d+-cp3\d+-","-py3-none-",u["filename"]).replace(pv,v) for u in prev["urls"]} @@ -65,9 +90,15 @@ if pv=="-": for plat in plats: hits=[f for f in have if re.search(r"-py3-none-",f) and re.search(plat,f)] want.update(hits) + if len(hits)==0 and plat=="win_amd64" and win_optional: + print("NOTE no windows wheel (optional for a native devN release)"); continue if len(hits)!=1: print("EXPECTED one wheel on %s, got %d"%(plat,len(hits))); gaps+=1 missing=sorted(want-set(have)); extra=sorted(set(have)-want) +if win_optional: + skipped=[m for m in missing if "win_amd64" in m] + for m in skipped: print("NOTE optional windows wheel not on PyPI:",m) + missing=[m for m in missing if m not in skipped] print("files on PyPI for %s: %d (previous release had %d)"%(v,len(have),len(prev["urls"]))) for m in missing: print("MISSING",m) for e in extra: print("EXTRA ",e) @@ -82,7 +113,7 @@ EOF # (c) sha256 of every file on PyPI against the checked wheels the workflow's verify job uploaded and the publish job sent if [ -n "$RUN" ] && [ -s "$W/pypi-new.json" ]; then mkdir -p "$W/artifacts" - gh run download "$RUN" -R "$R" -n release-dist -D "$W/artifacts" > /dev/null 2>&1 || fail "could not download the release-dist artifact of run $RUN (retention is 7 days)" + gh run download "$RUN" -R "$R" -n "$ARTIFACT" -D "$W/artifacts" > /dev/null 2>&1 || fail "could not download the $ARTIFACT artifact of run $RUN (retention is 7 days)" python3 - "$W/pypi-new.json" "$W/artifacts" << 'EOF' || FAILS=$((FAILS + 1)) import json,sys,hashlib,pathlib d=json.load(open(sys.argv[1])); art={} @@ -111,7 +142,11 @@ EOF ) read -r url sha name <<< "$pick" if [ -z "${url:-}" ]; then - fail "no single wheel for $plat on PyPI" + if [ "$plat" = 'win_amd64' ] && [ "$WIN_OPTIONAL" = 1 ]; then + printf 'SKIP no windows wheel on PyPI (optional for a native devN release)\n' + else + fail "no single wheel for $plat on PyPI" + fi continue fi mkdir -p "$W/dl" @@ -129,7 +164,28 @@ for py in 3.12 3.14; do fail "uv run --with $DIST==$V on $py: $(tail -1 "$W/install-$py.log")" continue } - ENG_SHA="$ENG_SHA" ENG_BUILD="$ENG_BUILD" V="$V" ${PIN:+taskset -c "$PIN"} "${UVRUN[@]}" python - << 'EOF' > "$W/smoke-$py.log" 2>&1 + if [ "$PACKAGE" = native ]; then + cat > "$W/smoke-native.py" << 'EOF' +import os, re, tempfile +import arcadedb as adb +assert adb.__version__ == os.environ["V"], adb.__version__ +assert adb.__backend__ == "native", adb.__backend__ +# A native wheel links the engine release its tag is named after: A.B.C for A.B.C and A.B.C.postN; a devN links DEV_ENGINE (any release). +engine = re.sub(r"\.post[0-9]+$", "", os.environ["V"]) +if ".dev" not in engine: assert adb.ENGINE_VERSION == engine, (adb.ENGINE_VERSION, engine) +print("engine", adb.ENGINE_VERSION, "backend", adb.__backend__) +with tempfile.TemporaryDirectory() as d: + with adb.create_database(os.path.join(d, "smoke")) as db: + db.command("sql", "CREATE VERTEX TYPE P") + with db.transaction(): + db.command("sql", "CREATE VERTEX P SET name = 'Ada'") + assert db.query("sql", "SELECT name FROM P").to_list()[0]["name"] == "Ada" + assert db.query("opencypher", "MATCH (n:P) RETURN n.name AS name").to_list()[0]["name"] == "Ada" +print("SMOKE OK", adb.__version__) +EOF + SMOKE="$W/smoke-native.py" + else + cat > "$W/smoke-jvm.py" << 'EOF' import hashlib, os, pathlib, re, tempfile, zipfile import arcadedb as adb assert adb.__version__ == os.environ["V"], adb.__version__ @@ -154,9 +210,13 @@ with tempfile.TemporaryDirectory() as d: assert db.query("opencypher", "MATCH (n:P) RETURN n.name AS name").to_list()[0]["name"] == "Ada" print("SMOKE OK", adb.__version__) EOF - if [ $? -eq 0 ] && grep -q 'SMOKE OK' "$W/smoke-$py.log"; then - pass "fresh environment Python $py: install, import, SQL, openCypher, official engine jar" - grep 'engine jar' "$W/smoke-$py.log" + SMOKE="$W/smoke-jvm.py" + fi + smoke_rc=0 + ENG_SHA="$ENG_SHA" ENG_BUILD="$ENG_BUILD" V="$V" ${PIN:+taskset -c "$PIN"} "${UVRUN[@]}" python -I "$SMOKE" > "$W/smoke-$py.log" 2>&1 || smoke_rc=$? + if [ "${smoke_rc:-0}" -eq 0 ] && grep -q 'SMOKE OK' "$W/smoke-$py.log"; then + pass "fresh environment Python $py: install, import, SQL, openCypher, engine ($PACKAGE)" + grep -E '^engine' "$W/smoke-$py.log" else fail "smoke on Python $py (see $W/smoke-$py.log)" tail -3 "$W/smoke-$py.log" diff --git a/scripts/bump_engine.py b/scripts/bump_engine.py new file mode 100644 index 0000000..e8103e8 --- /dev/null +++ b/scripts/bump_engine.py @@ -0,0 +1,260 @@ +#!/usr/bin/env python3 +"""Move the engine that dev, PR and nightly builds link to a new upstream release. + +After upstream releases the engine NEW (A.B.C), the pins below follow it, and the dev version moves +to the next base NEXT_DEV (A.B.C.dev0 of the next base, because a devN of a released base is refused): + + .github/workflows/test-native.yml DEV_ENGINE: "" + .github/workflows/test-native.yml DEFAULT_VERSION: "" + .github/workflows/test-native.yml the default of the manual-run input `version`: "" + native/pom.xml + native/scripts/build-image.sh ENGINE=${ARCADEDB_ENGINE_VERSION:-} + native/wheel/build_wheel.py the default of --version: "" + +A release tag derives its engine from the tag and needs none of these. Each pin is one exact +anchor: a missing or repeated anchor stops the script before it writes anything. + +native/wheel/THIRD_PARTY_NOTICES.md names the engine too, but it is generated from the build log +of the native image, so this script only reports it (see docs, "Release day"): the build fails +on that drift until the notices are regenerated. + +usage: + bump_engine.py --engine NEW --dev-version NEXT_DEV # edit the four pins + bump_engine.py --engine NEW --dev-version NEXT_DEV --check # report drift, write nothing + +NEXT_DEV must be a devN version that scripts/check_version.py would accept as the next version +(read from the git tags and PyPI of both packages, or from --published-file instead), and its +base must be higher than the base of NEW. + +Exit codes: 0 no drift (or edited), 1 drift found by --check or a refused version, 2 an anchor +or a source could not be read. Output is ASCII only. +""" + +from __future__ import annotations + +import argparse +import importlib.util +import re +import sys +from pathlib import Path, PurePosixPath + +SCRIPTS = Path(__file__).resolve().parent +PACKAGES = ("arcadedb-jvm", "arcadedb") +OWN_REMOTE = "https://github.com/ArcadeData/arcadedb-python.git" + +WORKFLOW = PurePosixPath(".github/workflows/test-native.yml") +POM = PurePosixPath("native/pom.xml") +BUILD_IMAGE = PurePosixPath("native/scripts/build-image.sh") +BUILD_WHEEL = PurePosixPath("native/wheel/build_wheel.py") +NOTICES = PurePosixPath("native/wheel/THIRD_PARTY_NOTICES.md") + +# (file, label, regex with one group around the value). One match each, exactly. +PINS = ( + (WORKFLOW, "DEV_ENGINE", r'^ DEV_ENGINE: "([^"]+)"[ \t]*$', "engine"), + (WORKFLOW, "DEFAULT_VERSION", r'^ DEFAULT_VERSION: "([^"]+)"[ \t]*$', "dev"), + ( + WORKFLOW, + "manual-run version default", + r'(?:Wheel version to build \(PEP 440: A\.B\.C\.devN, A\.B\.C or A\.B\.C\.postN\)"\n' + r' required: false\n default: ")([^"]+)"', + "dev", + ), + ( + POM, + "arcadedb.version", + r"([^<]+)", + "engine", + ), + ( + BUILD_IMAGE, + "ENGINE default", + r"^ENGINE=\$\{ARCADEDB_ENGINE_VERSION:-([^}]+)\}[ \t]*$", + "engine", + ), + ( + BUILD_WHEEL, + "build_wheel.py --version default", + r'^ "--version",\n default="([^"]+)",', + "dev", + ), +) +NOTICES_RE = re.compile(r"\(engine ([0-9][^,)]*),") + + +class BumpError(Exception): + """An anchor is missing or repeated, or a source could not be read.""" + + +def _load_check_version(): + spec = importlib.util.spec_from_file_location( + "check_version", SCRIPTS / "check_version.py" + ) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None # nosec B101 - spec of an existing file + spec.loader.exec_module(module) + return module + + +def validate_versions(engine: str, dev: str, published: list[str]) -> str | None: + """None when the pair is acceptable, else the reason it is refused.""" + cv = _load_check_version() + try: + e = cv.parse_version(engine) + d = cv.parse_version(dev) + except cv.VersionError as err: + return str(err) + if e.kind != "": + return f"--engine {engine} must be a release A.B.C (no devN or postN)" + if d.kind != "dev": + return f"--dev-version {dev} must be a devN version (A.B.C.devN)" + + def key(base: str) -> tuple[int, ...]: + return tuple(int(x) for x in base.split(".")) + + if key(d.base) <= key(e.base): + return ( + f"--dev-version {dev}: its base must be higher than the engine {engine}, " + "because a devN of a released base sorts before the release and is refused" + ) + try: + cv.check_next(dev, published) + except cv.VersionError as err: + return f"--dev-version refused by check_version.py: {err}" + return None + + +def published_versions(files: list[str], remote: str) -> list[str]: + cv = _load_check_version() + published: list[str] = [] + try: + if files: + for path in files: + published += cv.read_lines(path) + else: + for package in PACKAGES: + published += cv.pypi_versions(package) + published += cv.git_tag_versions(remote) + except OSError as err: + raise BumpError(str(err)) from err + return published + + +def plan(root: Path, engine: str, dev: str) -> list[tuple[Path, str, str, str, str]]: + """For each pin: (file, label, regex, current value, wanted value). Fails on a missing or repeated anchor.""" + rows = [] + for rel, label, pattern, kind in PINS: + try: + text = (root / rel).read_text(encoding="utf-8") + except OSError as err: + raise BumpError(f"cannot read {rel}: {err}") from err + found = re.findall(pattern, text, re.M) + if len(found) != 1: + raise BumpError( + f"{rel}: expected exactly one anchor for {label}, found {len(found)} " + "(the file changed: update PINS in scripts/bump_engine.py)" + ) + rows.append( + (rel, label, pattern, found[0], engine if kind == "engine" else dev) + ) + return rows + + +def apply(root: Path, rows) -> None: + """Write every pin that differs. All anchors were counted by plan() before this runs.""" + by_file: dict[Path, str] = {} + for rel, label, pattern, current, wanted in rows: + text = by_file.get(rel) or (root / rel).read_text(encoding="utf-8") + new, count = re.subn( + pattern, + lambda m, w=wanted: ( + m.group(0)[: m.start(1) - m.start(0)] + + w + + m.group(0)[m.end(1) - m.start(0) :] + ), + text, + flags=re.M, + ) + if count != 1: + raise BumpError(f"{rel}: {label} anchor matched {count} times on write") + by_file[rel] = new + for rel, text in by_file.items(): + (root / rel).write_text(text, encoding="utf-8") + + +def notices_engine(root: Path) -> str | None: + try: + text = (root / NOTICES).read_text(encoding="utf-8") + except OSError as err: + raise BumpError(f"cannot read {NOTICES}: {err}") from err + found = NOTICES_RE.findall(text) + return found[0] if len(set(found)) == 1 else None + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("--engine", required=True, help="the released engine A.B.C") + parser.add_argument( + "--dev-version", + required=True, + help="the next dev version, A.B.C.dev0 of the next base", + ) + parser.add_argument( + "--check", action="store_true", help="report drift, write nothing" + ) + parser.add_argument("--root", default=str(SCRIPTS.parent), help="repository root") + parser.add_argument( + "--published-file", + action="append", + default=[], + metavar="FILE", + help="published versions, one per line, read instead of PyPI and git tags (tests, offline)", + ) + parser.add_argument("--git-remote", default=OWN_REMOTE) + args = parser.parse_args(argv) + root = Path(args.root) + + try: + published = published_versions(args.published_file, args.git_remote) + reason = validate_versions(args.engine, args.dev_version, published) + if reason: + print(f"FAIL: {reason}") + return 1 + rows = plan(root, args.engine, args.dev_version) + notices = notices_engine(root) + except BumpError as err: + print(f"ERROR: {err}") + return 2 + + drift = 0 + for rel, label, _pattern, current, wanted in rows: + if current == wanted: + print(f"OK {rel} {label} = {wanted}") + else: + drift += 1 + print(f"DRIFT {rel} {label}: {current} -> {wanted}") + notices_drift = notices != args.engine + if notices_drift: + print( + f"TODO {NOTICES} names engine {notices or 'several'}, not {args.engine}: regenerate " + "it with native/wheel/gen_notices.py from the first build log of the new engine " + "(docs, Release day); this script does not edit it" + ) + else: + print(f"OK {NOTICES} names engine {args.engine}") + + if args.check: + return 1 if (drift or notices_drift) else 0 + if drift: + try: + apply(root, rows) + except BumpError as err: + print(f"ERROR: {err}") + return 2 + print(f"edited {drift} pin(s); review with git diff") + else: + print("nothing to edit") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/release_preflight.py b/scripts/release_preflight.py new file mode 100644 index 0000000..d27d44a --- /dev/null +++ b/scripts/release_preflight.py @@ -0,0 +1,677 @@ +#!/usr/bin/env python3 +"""Release-day preflight: is everything ready for the owner to push the tag VERSION? + +Read-only. Every check prints PASS, FAIL or SKIP with its evidence; the exit code is the number +of FAIL lines (0 means the tag may be pushed). It fails closed: a network or tool error is a +FAIL, never a PASS. It never tags, uploads, edits or writes anything in the repository. + +VERSION is the tag to push: A.B.C (final), A.B.C.postN (Python-only fix of a released engine) or +A.B.C.devN. The engine is the release the tag is named after (A.B.C), so for A.B.C.postN the +upstream checks (a) to (c) look at A.B.C. A devN tag is built from `latest` (JVM) and DEV_ENGINE +(native): upstream checks (a) to (c) are SKIPped for it. + +Checks: + (a) the GitHub release (not a draft or pre-release) of the engine exists on ArcadeData/arcadedb + (b) Maven Central has com.arcadedb:arcadedb-engine and arcadedb-server at the engine version + (the native build compiles them with GraalVM native-image) + (c) Docker Hub has arcadedata/arcadedb: for linux/amd64 and linux/arm64 + (the JVM wheels take their engine jars from that image) + (d) scripts/check_version.py accepts VERSION against the git tags and PyPI, for arcadedb-jvm + and for arcadedb + (e) the latest conclusive run of Test native, Test JVM and Test Python Examples on main is green + on every job + (f) the version and engine the release workflows derive from the tag are VERSION and the engine + (the real `run:` code of the workflow steps is executed with TAG=VERSION, not copied here) + (g) for a final or post tag: the repository variable ARCADEDB_NATIVE_READINESS_BAR_MET is true + +The two upstream channels (Maven Central, Docker Hub) are published by separate steps of the +upstream release and can lag each other: both must pass before the tag is pushed. + +usage: + release_preflight.py VERSION [--remote URL] [--repo OWNER/NAME] [--checkout DIR] + +Needs: network, git (for check_version.py), gh (logged in), bash on Linux or macOS (check f is a SKIP on Windows). +Tests: tests/test_release_preflight.py. Output is ASCII only. +""" + +from __future__ import annotations + +import argparse +import importlib.util +import json +import os +import re +import shutil +import subprocess # nosec B404 - runs git/gh/bash/python with fixed argument lists +import sys +import tempfile +import urllib.error +import urllib.request +from pathlib import Path +from typing import NamedTuple + +UPSTREAM = "ArcadeData/arcadedb" +OWN_REPO = "ArcadeData/arcadedb-python" +OWN_REMOTE = "https://github.com/ArcadeData/arcadedb-python.git" +MAVEN = "https://repo1.maven.org/maven2/com/arcadedb" +MAVEN_ARTIFACTS = ("arcadedb-engine", "arcadedb-server") +DOCKER_TAG_URL = "https://hub.docker.com/v2/repositories/arcadedata/arcadedb/tags" +PACKAGES = ("arcadedb-jvm", "arcadedb") +TEST_WORKFLOWS = ( + "test-native.yml", + "test-jvm.yml", + "test-python-examples.yml", +) +BAR_VARIABLE = "ARCADEDB_NATIVE_READINESS_BAR_MET" +HTTP_TIMEOUT = 30 +SCRIPTS = Path(__file__).resolve().parent + + +class Result(NamedTuple): + status: str # PASS, FAIL or SKIP + check: str + evidence: str + + +# ----- the two seams the tests replace: HTTP and subprocess ------------------------------- + + +def http_get(url: str, headers: dict[str, str] | None = None) -> tuple[int, str]: + """GET a fixed https URL. Returns (status, body); an HTTP error status is returned, not raised. + + Anything else (DNS, timeout, TLS) raises OSError, which the checks turn into a FAIL. + """ + if not url.startswith("https://"): + raise OSError(f"refusing URL {url}") + request = urllib.request.Request(url, headers=headers or {}) # noqa: S310 + try: + with urllib.request.urlopen( # nosec B310 - https URL checked above + request, timeout=HTTP_TIMEOUT + ) as response: + return response.status, response.read().decode("utf-8", "replace") + except urllib.error.HTTPError as err: + return err.code, err.read().decode("utf-8", "replace") + except (urllib.error.URLError, TimeoutError, ValueError) as err: + raise OSError(f"cannot read {url}: {err}") from err + + +def run( + cmd: list[str], env: dict[str, str] | None = None, cwd: str | None = None +) -> tuple[int, str, str]: + """Run a command with a fixed argument list. Returns (returncode, stdout, stderr).""" + try: + done = subprocess.run( # nosec B603 - fixed argument list, no shell + cmd, + capture_output=True, + text=True, + timeout=300, + env=env, + cwd=cwd, + check=False, + ) + except (subprocess.SubprocessError, OSError) as err: + raise OSError(f"cannot run {cmd[0]}: {err}") from err + return done.returncode, done.stdout, done.stderr + + +def _load_check_version(): + spec = importlib.util.spec_from_file_location( + "check_version", SCRIPTS / "check_version.py" + ) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None # nosec B101 - spec of an existing file + spec.loader.exec_module(module) + return module + + +# ----- the version and the engine --------------------------------------------------------- + + +def engine_of(version: str) -> str | None: + """The engine release a tag is named after: A.B.C for A.B.C and A.B.C.postN, None for a devN.""" + cv = _load_check_version() + parsed = cv.parse_version(version) + return None if parsed.kind == "dev" else parsed.base + + +def _http_error(check: str, err: OSError) -> Result: + return Result("FAIL", check, f"cannot read the source (fail closed): {err}") + + +# ----- (a) upstream release --------------------------------------------------------------- + + +def check_upstream_release(engine: str) -> list[Result]: + check = f"(a) GitHub release {engine} on {UPSTREAM}" + base = f"https://api.github.com/repos/{UPSTREAM}" + headers = {"Accept": "application/vnd.github+json"} + if os.environ.get("GITHUB_TOKEN"): + headers["Authorization"] = f"Bearer {os.environ['GITHUB_TOKEN']}" + try: + status, body = http_get(f"{base}/releases/tags/{engine}", headers) + if status == 200: + data = json.loads(body) + if data.get("tag_name") != engine: + return [ + Result("FAIL", check, f"release has tag {data.get('tag_name')!r}") + ] + if data.get("draft") or data.get("prerelease"): + flags = [k for k in ("draft", "prerelease") if data.get(k)] + return [Result("FAIL", check, f"release is marked {'/'.join(flags)}")] + return [ + Result( + "PASS", + check, + f"published {data.get('published_at')} {data.get('html_url')}", + ) + ] + if status != 404: + return [Result("FAIL", check, f"GitHub answered HTTP {status}")] + status, _ = http_get(f"{base}/git/ref/tags/{engine}", headers) + if status == 200: + return [ + Result( + "FAIL", check, "the tag exists but there is no published release" + ) + ] + if status == 404: + return [Result("FAIL", check, "neither a release nor a tag of that name")] + return [Result("FAIL", check, f"GitHub answered HTTP {status} for the tag")] + except (OSError, ValueError) as err: + return [_http_error(check, OSError(str(err)))] + + +# ----- (b) Maven Central ------------------------------------------------------------------ + + +def check_maven(engine: str) -> list[Result]: + results = [] + for artifact in MAVEN_ARTIFACTS: + url = f"{MAVEN}/{artifact}/{engine}/{artifact}-{engine}.pom" + check = f"(b) Maven Central com.arcadedb:{artifact}:{engine}" + try: + status, _ = http_get(url) + except OSError as err: + results.append(_http_error(check, err)) + continue + if status == 200: + results.append(Result("PASS", check, f"HTTP 200 {url}")) + elif status == 404: + results.append(Result("FAIL", check, f"HTTP 404 {url} (not published yet)")) + else: + results.append(Result("FAIL", check, f"HTTP {status} {url}")) + return results + + +# ----- (c) Docker Hub --------------------------------------------------------------------- + + +def check_docker(engine: str) -> list[Result]: + check = f"(c) Docker Hub arcadedata/arcadedb:{engine}" + url = f"{DOCKER_TAG_URL}/{engine}" + try: + status, body = http_get(url) + if status == 404: + return [Result("FAIL", check, f"HTTP 404 {url} (not published yet)")] + if status != 200: + return [Result("FAIL", check, f"HTTP {status} {url}")] + data = json.loads(body) + except (OSError, ValueError) as err: + return [_http_error(check, OSError(str(err)))] + if data.get("name") != engine: + return [Result("FAIL", check, f"the tag answers as {data.get('name')!r}")] + archs = sorted( + { + image.get("architecture") + for image in data.get("images", []) + if image.get("architecture") and image.get("architecture") != "unknown" + } + ) + missing = [a for a in ("amd64", "arm64") if a not in archs] + if missing: + return [ + Result( + "FAIL", + check, + f"architectures {','.join(archs) or 'none'}; missing {','.join(missing)}", + ) + ] + return [ + Result( + "PASS", + check, + f"architectures {','.join(archs)}, last updated {data.get('last_updated')}", + ) + ] + + +# ----- (d) version rule ------------------------------------------------------------------- + + +def check_version_rule(version: str, remote: str) -> list[Result]: + results = [] + for package in PACKAGES: + check = f"(d) check_version.py {version} for {package}" + cmd = [ + sys.executable, + "-I", + str(SCRIPTS / "check_version.py"), + version, + "--git-remote", + remote, + "--pypi", + package, + ] + try: + code, out, err = run(cmd) + except OSError as err_: + results.append(_http_error(check, err_)) + continue + line = (out.strip().splitlines() or [err.strip() or "no output"])[-1] + if code == 0: + results.append(Result("PASS", check, line)) + elif code == 1: + results.append(Result("FAIL", check, line)) + else: + results.append( + Result( + "FAIL", check, f"exit {code}, a source could not be read: {line}" + ) + ) + return results + + +# ----- (e) CI on main --------------------------------------------------------------------- + + +def _gh_json(args: list[str]): + code, out, err = run(["gh", *args]) + if code != 0: + raise OSError(f"gh {' '.join(args[:2])} failed: {err.strip() or out.strip()}") + try: + return json.loads(out) + except ValueError as exc: + raise OSError(f"gh {' '.join(args[:2])} printed no JSON") from exc + + +def check_ci(repo: str) -> list[Result]: + results = [] + try: + tip = _gh_json(["api", f"repos/{repo}/commits/main"]).get("sha", "") + except OSError: + tip = "" # only used for a note + for workflow in TEST_WORKFLOWS: + check = f"(e) latest conclusive {workflow} run on main" + try: + runs = _gh_json( + [ + "run", + "list", + "-R", + repo, + "--workflow", + workflow, + "--branch", + "main", + "--limit", + "20", + "--json", + "databaseId,status,conclusion,event,headSha,createdAt", + ] + ) + # A cancelled run was replaced by a newer commit; the latest run that reached a verdict counts. + verdicts = [ + r + for r in runs + if r.get("status") == "completed" + and r.get("conclusion") not in ("cancelled", "skipped") + ] + if not verdicts: + results.append(Result("FAIL", check, "no completed run on main found")) + continue + latest = verdicts[0] + jobs = _gh_json( + [ + "run", + "view", + str(latest["databaseId"]), + "-R", + repo, + "--json", + "jobs", + ] + ).get("jobs", []) + except (OSError, KeyError) as err: + results.append(_http_error(check, OSError(str(err)))) + continue + counts: dict[str, int] = {} + for job in jobs: + key = job.get("conclusion") or job.get("status") or "unknown" + counts[key] = counts.get(key, 0) + 1 + bad = [ + f"{j.get('name')} ({j.get('conclusion') or j.get('status')})" + for j in jobs + if j.get("conclusion") not in ("success", "skipped") + ] + sha = str(latest.get("headSha", "")) + note = "" + if tip and sha and tip != sha: + note = f"; main is now at {tip[:12]}, this run is for an earlier commit" + newer = runs[: runs.index(latest)] + if newer: + note += "; newer runs: " + ",".join( + f"{r.get('status')}/{r.get('conclusion') or '-'}" for r in newer[:3] + ) + evidence = ( + f"run {latest['databaseId']} ({latest.get('event')}, {latest.get('createdAt')}, " + f"sha {sha[:12]}) jobs " + + ", ".join(f"{n} {k}" for k, n in sorted(counts.items())) + + note + ) + if latest.get("conclusion") != "success" or bad or not jobs: + detail = "; not green: " + ", ".join(bad[:6]) if bad else "" + if not jobs: + detail = "; the run lists no jobs" + results.append(Result("FAIL", check, evidence + detail)) + elif counts.get("success", 0) == 0: + results.append(Result("FAIL", check, evidence + "; no job succeeded")) + else: + results.append(Result("PASS", check, evidence)) + return results + + +# ----- (f) what the release workflows derive from the tag --------------------------------- + + +def extract_step_run(text: str, step_name: str) -> str | None: + """The shell code of the `run: |` block of the step `- name: `, dedented, or None.""" + lines = text.splitlines() + for i, line in enumerate(lines): + m = re.match(r"^(\s*)- name: " + re.escape(step_name) + r"\s*$", line) + if not m: + continue + step_indent = len(m.group(1)) + for j in range(i + 1, len(lines)): + current = lines[j] + if current.strip() and len(current) - len(current.lstrip()) <= step_indent: + return None # the next step started before a run block + r = re.match(r"^(\s*)run: \|\s*$", current) + if not r: + continue + run_indent = len(r.group(1)) + body = [] + for k in range(j + 1, len(lines)): + nxt = lines[k] + if nxt.strip() and len(nxt) - len(nxt.lstrip()) <= run_indent: + break + body.append(nxt) + block_indent = min( + (len(b) - len(b.lstrip()) for b in body if b.strip()), default=0 + ) + return "\n".join(b[block_indent:] for b in body) + "\n" + return None + return None + + +def _env_values(text: str, names: tuple[str, ...]) -> dict[str, str]: + values = {} + for name in names: + m = re.search(r"^\s{2}" + name + r': "([^"]*)"\s*$', text, re.M) + if m: + values[name] = m.group(1) + return values + + +def _run_step(script: str, env: dict[str, str]) -> dict[str, str]: + """Run a workflow step's shell code under bash and return what it wrote to GITHUB_OUTPUT.""" + with tempfile.TemporaryDirectory() as tmp: + out_file = Path(tmp) / "github_output" + out_file.write_text("", encoding="utf-8") + full_env = { + "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "GITHUB_OUTPUT": str(out_file), + **env, + } + code, _, err = run(["bash", "-c", script], env=full_env, cwd=tmp) + if code != 0: + raise ValueError(f"the step exited {code}: {err.strip().splitlines()[-1:]}") + pairs = {} + for line in out_file.read_text(encoding="utf-8").splitlines(): + if "=" in line: + key, value = line.split("=", 1) + pairs[key] = value + return pairs + + +def posix_shell_unavailable() -> str | None: + """The reason check (f) cannot run here, or None. The workflow steps are bash code for a POSIX shell.""" + if os.name == "nt": + return "Windows: the workflow step code needs a POSIX shell; run the preflight on Linux or macOS" + if shutil.which("bash") is None: + return "bash not found" + return None + + +def check_derivation(version: str, repo_root: Path) -> list[Result]: + cv = _load_check_version() + parsed = cv.parse_version(version) + engine = None if parsed.kind == "dev" else parsed.base + final = parsed.kind != "dev" + unavailable = posix_shell_unavailable() + if unavailable: + return [ + Result( + "SKIP", + "(f) workflow derivation of version and engine", + unavailable, + ) + ] + workflows = repo_root / ".github" / "workflows" + results = [] + + def step(file: str, name: str) -> tuple[str | None, str]: + try: + text = (workflows / file).read_text(encoding="utf-8") + except OSError as err: + return None, f"cannot read {file}: {err}" + script = extract_step_run(text, name) + if script is None: + return ( + None, + f"{file} has no step {name!r} with a run block (the workflow changed: update this check)", + ) + return script, text + + # release-jvm.yml: image tag the JVM wheels take their jars from + check = f"(f) release-jvm.yml derives, for tag {version}" + script, text = step("release-jvm.yml", "Validate tag") + if script is None: + results.append(Result("FAIL", check, text)) + else: + try: + got = _run_step(script, {"TAG": version}) + want_image = engine if engine else "latest" + problems = [] + if got.get("version") != version: + problems.append(f"version {got.get('version')!r}") + if got.get("arcadedb-tag") != want_image: + problems.append( + f"image tag {got.get('arcadedb-tag')!r}, expected {want_image!r}" + ) + if engine and got.get("base-version") != engine: + problems.append( + f"engine {got.get('base-version')!r}, expected {engine!r}" + ) + evidence = ( + f"version {got.get('version')}, engine {got.get('base-version')}, " + f"image arcadedata/arcadedb:{got.get('arcadedb-tag')}" + ) + results.append( + Result("FAIL", check, evidence + "; wrong: " + ", ".join(problems)) + if problems + else Result("PASS", check, evidence) + ) + except (ValueError, OSError) as err: + results.append(Result("FAIL", check, str(err))) + + # release-native.yml: the tag is accepted and its final flag + check = f"(f) release-native.yml derives, for tag {version}" + script, text = step("release-native.yml", "Validate tag") + if script is None: + results.append(Result("FAIL", check, text)) + else: + try: + got = _run_step(script, {"TAG": version}) + evidence = f"version {got.get('version')}, final {got.get('final')}" + problems = [] + if got.get("version") != version: + problems.append(f"version {got.get('version')!r}") + if got.get("final") != ("true" if final else "false"): + problems.append(f"final {got.get('final')!r}") + results.append( + Result("FAIL", check, evidence + "; wrong: " + ", ".join(problems)) + if problems + else Result("PASS", check, evidence) + ) + except (ValueError, OSError) as err: + results.append(Result("FAIL", check, str(err))) + + # test-native.yml: the engine the native image links + check = f"(f) test-native.yml links, for version {version}" + script, text = step("test-native.yml", "Resolve the version and the engine") + if script is None: + results.append(Result("FAIL", check, text)) + else: + env = {"INPUT_VERSION": version, "INPUT_ENGINE": ""} + env.update(_env_values(text, ("DEFAULT_VERSION", "DEV_ENGINE"))) + try: + got = _run_step(script, env) + evidence = f"engine {got.get('engine')}, final {got.get('final')}" + if not final: + evidence += " (devN: DEV_ENGINE)" + problems = [] + if got.get("version") != version: + problems.append(f"version {got.get('version')!r}") + if got.get("final") != ("true" if final else "false"): + problems.append(f"final {got.get('final')!r}") + if engine and got.get("engine") != engine: + problems.append(f"engine {got.get('engine')!r}, expected {engine!r}") + results.append( + Result("FAIL", check, evidence + "; wrong: " + ", ".join(problems)) + if problems + else Result("PASS", check, evidence) + ) + except (ValueError, OSError) as err: + results.append(Result("FAIL", check, str(err))) + return results + + +# ----- (g) readiness bar ------------------------------------------------------------------ + + +def check_readiness_bar(version: str, repo: str) -> list[Result]: + check = f"(g) repository variable {BAR_VARIABLE}" + if re.search(r"\.dev[0-9]+$", version): + return [Result("SKIP", check, "a devN native release does not need the bar")] + try: + code, out, err = run( + [ + "gh", + "api", + f"repos/{repo}/actions/variables/{BAR_VARIABLE}", + "--jq", + ".value", + ] + ) + except OSError as exc: + return [_http_error(check, exc)] + if code == 0: + value = out.strip() + if value == "true": + return [Result("PASS", check, "true")] + return [ + Result( + "FAIL", + check, + f"value is {value!r}: release-native.yml publishes no final wheel until a maintainer sets it to true", + ) + ] + if "404" in err: + return [ + Result( + "FAIL", + check, + "the variable is not set: release-native.yml publishes no final wheel", + ) + ] + return [ + Result( + "SKIP", + check, + f"cannot read repository variables with this gh login ({err.strip()[:120]}); the owner checks it", + ) + ] + + +# ----- main ------------------------------------------------------------------------------- + + +def run_checks(version: str, remote: str, repo: str, repo_root: Path) -> list[Result]: + cv = _load_check_version() + try: + cv.parse_version(version) + except cv.VersionError as err: + return [Result("FAIL", "(0) version syntax", str(err))] + engine = engine_of(version) + results: list[Result] = [] + if engine is None: + for letter, what in ( + ("a", "GitHub release"), + ("b", "Maven Central"), + ("c", "Docker Hub"), + ): + results.append( + Result( + "SKIP", + f"({letter}) upstream {what}", + "a devN tag is built from latest (JVM) and DEV_ENGINE (native), not from a release", + ) + ) + else: + results += check_upstream_release(engine) + results += check_maven(engine) + results += check_docker(engine) + results += check_version_rule(version, remote) + results += check_ci(repo) + results += check_derivation(version, repo_root) + results += check_readiness_bar(version, repo) + return results + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument( + "version", help="the tag to push (A.B.C, A.B.C.postN or A.B.C.devN)" + ) + parser.add_argument( + "--remote", default=OWN_REMOTE, help="git remote read for the tags" + ) + parser.add_argument( + "--repo", default=OWN_REPO, help="GitHub repository of the CI runs" + ) + parser.add_argument("--checkout", dest="repo_root", default=str(SCRIPTS.parent)) + args = parser.parse_args(argv) + + results = run_checks(args.version, args.remote, args.repo, Path(args.repo_root)) + for r in results: + print(f"{r.status:<5} {r.check}: {r.evidence}") + fails = sum(1 for r in results if r.status == "FAIL") + skips = sum(1 for r in results if r.status == "SKIP") + print(f"\n{fails} FAIL(s), {skips} SKIP(s) for {args.version}") + if fails == 0: + print("Ready: the owner may push the tag (the only publishing step).") + return fails + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/conftest.py b/tests/conftest.py index 7669916..01ab332 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -32,6 +32,9 @@ # runs scripts/stage_jars.sh under bash with a fake docker; the bash a Windows # subprocess finds may be WSL's rather than Git Bash "test_stage_jars.py": "bash with a fake docker", + # the release preflight is a maintainer tool for a POSIX shell: its check (f) runs the workflows' bash step code, + # and on Windows it reports a SKIP, which its own tests cover on Linux and macOS (ArcadeData/arcadedb-python#208) + "test_release_preflight.py": "runs the workflows' bash step code", } if sys.platform == "win32": collect_ignore.extend(WINDOWS_NOT_APPLICABLE) diff --git a/tests/test_bump_engine.py b/tests/test_bump_engine.py new file mode 100644 index 0000000..9538fb6 --- /dev/null +++ b/tests/test_bump_engine.py @@ -0,0 +1,207 @@ +"""scripts/bump_engine.py: the four engine pins move together, drift is reported, a bad NEXT_DEV is refused. + +The tests run on fixture copies of the real files, so an anchor that drifts in the repository fails here. +The versions are made-up ones (99.x), never a real release. +""" + +from __future__ import annotations + +import importlib.util +import shutil +import sys +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parents[1] +SCRIPT = REPO / "scripts" / "bump_engine.py" +FIXTURE_FILES = ( + ".github/workflows/test-native.yml", + "native/pom.xml", + "native/scripts/build-image.sh", + "native/wheel/THIRD_PARTY_NOTICES.md", + "native/wheel/build_wheel.py", +) + + +def _load(): + spec = importlib.util.spec_from_file_location("bump_engine", SCRIPT) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None + sys.modules["bump_engine"] = module + spec.loader.exec_module(module) + return module + + +be = _load() + + +@pytest.fixture +def tree(tmp_path): + for rel in FIXTURE_FILES: + dst = tmp_path / rel + dst.parent.mkdir(parents=True, exist_ok=True) + shutil.copyfile(REPO / rel, dst) + published = tmp_path / "published.txt" + published.write_text("98.1.1\n98.1.1.dev0\n98.12.1\n99.1.0\n", encoding="utf-8") + return tmp_path + + +def _run(tree, *extra, engine="99.1.1", dev="99.2.1.dev0"): + return be.main( + [ + "--engine", + engine, + "--dev-version", + dev, + "--root", + str(tree), + "--published-file", + str(tree / "published.txt"), + *extra, + ] + ) + + +def _snapshot(tree): + return {rel: (tree / rel).read_bytes() for rel in FIXTURE_FILES} + + +def test_edit_moves_all_the_pins(tree, capsys): + assert _run(tree) == 0 + workflow = (tree / ".github/workflows/test-native.yml").read_text() + assert ' DEV_ENGINE: "99.1.1"\n' in workflow + assert ' DEFAULT_VERSION: "99.2.1.dev0"\n' in workflow + assert 'required: false\n default: "99.2.1.dev0"' in workflow + assert ( + '"--version",\n default="99.2.1.dev0",' + in (tree / "native/wheel/build_wheel.py").read_text() + ) + assert ( + "99.1.1" + in (tree / "native/pom.xml").read_text() + ) + assert ( + "${ARCADEDB_ENGINE_VERSION:-99.1.1}" + in (tree / "native/scripts/build-image.sh").read_text() + ) + out = capsys.readouterr().out + assert out.count("DRIFT") == 6 + assert "TODO" in out # the notices follow the engine and are not edited here + # nothing else changed: only those lines differ from the real files + for rel in FIXTURE_FILES[:3]: + old = (REPO / rel).read_text().splitlines() + new = (tree / rel).read_text().splitlines() + assert len(old) == len(new) + assert sum(a != b for a, b in zip(old, new)) in (1, 3) + assert (tree / "native/wheel/THIRD_PARTY_NOTICES.md").read_bytes() == ( + REPO / "native/wheel/THIRD_PARTY_NOTICES.md" + ).read_bytes() + + +def test_second_run_is_a_no_op(tree, capsys): + assert _run(tree) == 0 + after = _snapshot(tree) + assert _run(tree) == 0 + assert _snapshot(tree) == after + assert "nothing to edit" in capsys.readouterr().out + + +def test_check_reports_drift_and_writes_nothing(tree, capsys): + before = _snapshot(tree) + assert _run(tree, "--check") == 1 + assert _snapshot(tree) == before + assert "DRIFT" in capsys.readouterr().out + + +def test_check_still_fails_while_the_notices_name_another_engine(tree, capsys): + assert _run(tree) == 0 + capsys.readouterr() + assert _run(tree, "--check") == 1 # pins match, notices do not + out = capsys.readouterr().out + assert "DRIFT" not in out + assert "TODO" in out + notices = tree / "native/wheel/THIRD_PARTY_NOTICES.md" + notices.write_text(_with_engine(notices.read_text(), "99.1.1"), encoding="utf-8") + assert _run(tree, "--check") == 0 + + +def _with_engine(text: str, engine: str) -> str: + return be.NOTICES_RE.sub( + lambda m: m.group(0).replace(m.group(1), engine), text, count=1 + ) + + +@pytest.mark.parametrize( + "engine, dev, why", + [ + ("99.1.1.dev0", "99.2.1.dev0", "must be a release"), + ("99.1.1.post1", "99.2.1.dev0", "must be a release"), + ("99.1.1", "99.2.1", "must be a devN"), + ("99.1.1", "99.1.1.dev0", "higher than the engine"), + ("99.1.1", "98.5.1.dev0", "higher than the engine"), + ("99.1.1", "99.2.1.dev3", "next dev version of 99.2.1 is dev0"), + ("99.1.1", "99.2.1.rc1", "not X.Y.Z"), + ("99.1", "99.2.1.dev0", "not X.Y.Z"), + ], +) +def test_refused_versions_write_nothing(tree, capsys, engine, dev, why): + before = _snapshot(tree) + assert _run(tree, engine=engine, dev=dev) == 1 + assert _snapshot(tree) == before + assert why in capsys.readouterr().out + + +def test_dev_version_of_a_published_base_is_refused(tree, capsys): + (tree / "published.txt").write_text("99.2.1\n", encoding="utf-8") + before = _snapshot(tree) + assert _run(tree) == 1 + assert "already released" in capsys.readouterr().out + assert _snapshot(tree) == before + + +@pytest.mark.parametrize( + "rel, old, new", + [ + (FIXTURE_FILES[0], "DEV_ENGINE:", "DEV_ENGINE_RENAMED:"), + (FIXTURE_FILES[0], "DEFAULT_VERSION:", "DEFAULT_RENAMED:"), + ( + FIXTURE_FILES[0], + "Wheel version to build (PEP 440: A.B.C.devN,", + "Wheel version (PEP 440: A.B.C.devN,", + ), + (FIXTURE_FILES[1], "", ""), + (FIXTURE_FILES[2], "ARCADEDB_ENGINE_VERSION:-", "ARCADEDB_ENGINE:-"), + (FIXTURE_FILES[4], '"--version",', '"--wheel-version",'), + ], +) +def test_missing_anchor_fails_loudly_and_writes_nothing(tree, capsys, rel, old, new): + path = tree / rel + text = path.read_text() + assert old in text + path.write_text(text.replace(old, new)) + before = _snapshot(tree) + assert _run(tree) == 2 + assert _snapshot(tree) == before + assert "exactly one anchor" in capsys.readouterr().out + + +def test_repeated_anchor_fails_loudly(tree, capsys): + path = tree / "native/pom.xml" + path.write_text(path.read_text() + "1.0.0\n") + before = _snapshot(tree) + assert _run(tree) == 2 + assert _snapshot(tree) == before + assert "found 2" in capsys.readouterr().out + + +def test_missing_file_is_an_error(tree, capsys): + (tree / "native/pom.xml").unlink() + assert _run(tree) == 2 + assert "cannot read native/pom.xml" in capsys.readouterr().out + + +def test_published_file_read_error_is_exit_2(tree, capsys): + (tree / "published.txt").unlink() + assert _run(tree) == 2 + assert "ERROR" in capsys.readouterr().out diff --git a/tests/test_release_preflight.py b/tests/test_release_preflight.py new file mode 100644 index 0000000..8edc978 --- /dev/null +++ b/tests/test_release_preflight.py @@ -0,0 +1,478 @@ +"""scripts/release_preflight.py: every check passes on good data and fails on each way it can go wrong. + +HTTP and subprocess are replaced by fakes, so nothing here touches the network, gh or git. The one +real process is bash, which runs the shell code of the workflow steps (check f) on the real workflow +files. The versions are made-up ones (99.x), never a real release. +""" + +from __future__ import annotations + +import importlib.util +import json +import os +import sys +import types +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parents[1] +SCRIPT = REPO / "scripts" / "release_preflight.py" + + +def _load(): + spec = importlib.util.spec_from_file_location("release_preflight", SCRIPT) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None + sys.modules["release_preflight"] = module + spec.loader.exec_module(module) + return module + + +rp = _load() +REAL_RUN = rp.run +V = "99.1.1" +# Check (f) runs the bash code of the workflow steps. This file is not collected on Windows (tests/conftest.py, +# WINDOWS_NOT_APPLICABLE); on Windows the preflight reports a SKIP for (f), which the last two tests cover here. +JOBS_OK = { + "jobs": [ + {"name": "build", "conclusion": "success"}, + {"name": "x", "conclusion": "skipped"}, + ] +} + + +class Net: + """A fake world: the answers of GitHub, Maven Central, Docker Hub, gh and check_version.py.""" + + def __init__(self, version=V): + self.version = version + self.http = {} # url -> (status, body) or an OSError + self.runs = [ + { + "databaseId": 7, + "status": "completed", + "conclusion": "success", + "event": "schedule", + "headSha": "a" * 40, + "createdAt": "t", + } + ] + self.jobs = {7: JOBS_OK} + self.check_version = { + "arcadedb-jvm": (0, "OK: fine\n", ""), + "arcadedb": (0, "OK: fine\n", ""), + } + self.gh_variable = (0, "true\n", "") + self.gh_error = None + engine = version.split(".post")[0] + self.http[ + f"https://api.github.com/repos/ArcadeData/arcadedb/releases/tags/{engine}" + ] = ( + 200, + json.dumps( + { + "tag_name": engine, + "draft": False, + "prerelease": False, + "published_at": "2099-01-01", + "html_url": "https://example/r", + } + ), + ) + for artifact in ("arcadedb-engine", "arcadedb-server"): + self.http[ + f"https://repo1.maven.org/maven2/com/arcadedb/{artifact}/{engine}/{artifact}-{engine}.pom" + ] = (200, "") + self.http[ + f"https://hub.docker.com/v2/repositories/arcadedata/arcadedb/tags/{engine}" + ] = ( + 200, + json.dumps( + { + "name": engine, + "last_updated": "2099-01-01", + "images": [ + {"architecture": "amd64"}, + {"architecture": "arm64"}, + {"architecture": "unknown"}, + ], + } + ), + ) + + def http_get(self, url, headers=None): + answer = self.http.get(url, (404, "")) + if isinstance(answer, OSError): + raise answer + return answer + + def run(self, cmd, env=None, cwd=None): + if cmd[0] == "bash": + return REAL_RUN(cmd, env=env, cwd=cwd) + if self.gh_error: + raise OSError(self.gh_error) + if cmd[0] == "gh" and cmd[1] == "api" and "commits/main" in cmd[2]: + return 0, json.dumps({"sha": "a" * 40}), "" + if cmd[0] == "gh" and cmd[1] == "api": + return self.gh_variable + if cmd[:3] == ["gh", "run", "list"]: + return 0, json.dumps(self.runs), "" + if cmd[:3] == ["gh", "run", "view"]: + return 0, json.dumps(self.jobs[int(cmd[3])]), "" + if "check_version.py" in " ".join(cmd): + return self.check_version[cmd[-1]] + raise AssertionError(f"unexpected command {cmd}") + + +@pytest.fixture +def net(monkeypatch): + n = Net() + monkeypatch.setattr(rp, "http_get", n.http_get) + monkeypatch.setattr(rp, "run", n.run) + return n + + +def _checks(version=V, root=REPO): + return rp.run_checks(version, "remote", "owner/repo", root) + + +def _fails(results): + return [r for r in results if r.status == "FAIL"] + + +def _only_fail(results, prefix): + fails = _fails(results) + assert [r.check[: len(prefix)] for r in fails] == [prefix], fails + return fails[0] + + +@pytest.mark.parametrize("version", [V, V + ".post2", "99.1.1"]) +def test_everything_ready_is_zero_fails(net, version): + net.__init__(version) + results = _checks(version) + assert not _fails(results), _fails(results) + assert {r.check[:3] for r in results} >= { + "(a)", + "(b)", + "(c)", + "(d)", + "(e)", + "(f)", + "(g)", + } + + +def test_dev_version_skips_the_upstream_checks(net): + results = _checks("99.1.1.dev0") + skipped = [r.check[:3] for r in results if r.status == "SKIP"] + assert skipped[:3] == ["(a)", "(b)", "(c)"] and "(g)" in skipped + assert not _fails(results) + + +def test_post_version_checks_the_engine_it_is_named_after(net): + net.__init__("99.1.1.post3") + results = _checks("99.1.1.post3") + assert any("release 99.1.1 on" in r.check for r in results) + assert not _fails(results) + + +@pytest.mark.parametrize("version", ["v99.1.1", "99.1.1rc1", "99.1.1.post0", "99.1"]) +def test_bad_version_syntax_fails_alone(net, version): + results = _checks(version) + assert [r.check for r in results] == ["(0) version syntax"] and results[ + 0 + ].status == "FAIL" + + +# (a) +def test_a_missing_release_and_tag(net): + net.http.clear() + r = _fails(_checks())[0] + assert r.check.startswith("(a)") and "neither a release nor a tag" in r.evidence + + +def test_a_tag_without_release(net): + del net.http[f"https://api.github.com/repos/ArcadeData/arcadedb/releases/tags/{V}"] + net.http[f"https://api.github.com/repos/ArcadeData/arcadedb/git/ref/tags/{V}"] = ( + 200, + "{}", + ) + assert "no published release" in _only_fail(_checks(), "(a)").evidence + + +@pytest.mark.parametrize("flag", ["draft", "prerelease"]) +def test_a_draft_or_prerelease(net, flag): + url = f"https://api.github.com/repos/ArcadeData/arcadedb/releases/tags/{V}" + body = json.loads(net.http[url][1]) + body[flag] = True + net.http[url] = (200, json.dumps(body)) + assert flag in _only_fail(_checks(), "(a)").evidence + + +def test_a_http_error_and_network_error_fail_closed(net): + url = f"https://api.github.com/repos/ArcadeData/arcadedb/releases/tags/{V}" + net.http[url] = (500, "") + assert "HTTP 500" in _only_fail(_checks(), "(a)").evidence + net.http[url] = OSError("timed out") + assert "fail closed" in _only_fail(_checks(), "(a)").evidence + + +# (b) +@pytest.mark.parametrize("artifact", ["arcadedb-engine", "arcadedb-server"]) +def test_b_missing_artifact(net, artifact): + del net.http[ + f"https://repo1.maven.org/maven2/com/arcadedb/{artifact}/{V}/{artifact}-{V}.pom" + ] + r = _only_fail(_checks(), "(b)") + assert artifact in r.check and "not published yet" in r.evidence + + +def test_b_errors_fail_closed(net): + url = f"https://repo1.maven.org/maven2/com/arcadedb/arcadedb-server/{V}/arcadedb-server-{V}.pom" + net.http[url] = (503, "") + assert "HTTP 503" in _only_fail(_checks(), "(b)").evidence + net.http[url] = OSError("dns") + assert "fail closed" in _only_fail(_checks(), "(b)").evidence + + +# (c) +def test_c_missing_tag(net): + del net.http[f"https://hub.docker.com/v2/repositories/arcadedata/arcadedb/tags/{V}"] + assert "not published yet" in _only_fail(_checks(), "(c)").evidence + + +def test_c_missing_architecture(net): + url = f"https://hub.docker.com/v2/repositories/arcadedata/arcadedb/tags/{V}" + net.http[url] = ( + 200, + json.dumps({"name": V, "images": [{"architecture": "amd64"}]}), + ) + assert "missing arm64" in _only_fail(_checks(), "(c)").evidence + + +def test_c_wrong_name_bad_json_and_network_error(net): + url = f"https://hub.docker.com/v2/repositories/arcadedata/arcadedb/tags/{V}" + net.http[url] = (200, json.dumps({"name": "latest", "images": []})) + assert "answers as" in _only_fail(_checks(), "(c)").evidence + net.http[url] = (200, "not json") + assert _only_fail(_checks(), "(c)") + net.http[url] = OSError("tls") + assert "fail closed" in _only_fail(_checks(), "(c)").evidence + + +# (d) +@pytest.mark.parametrize("code", [1, 2]) +def test_d_check_version_refuses(net, code): + net.check_version["arcadedb"] = (code, "FAIL: nope\n", "") + r = _only_fail(_checks(), "(d)") + assert "for arcadedb:" in r.check or r.check.endswith("for arcadedb") + assert "nope" in r.evidence + + +def test_d_both_packages_are_checked(net): + results = [r for r in _checks() if r.check.startswith("(d)")] + assert [r.check.rsplit(" ", 1)[1] for r in results] == ["arcadedb-jvm", "arcadedb"] + + +# (e) +def test_e_failed_job(net): + net.jobs[7] = { + "jobs": [ + {"name": "build mac", "conclusion": "failure"}, + {"name": "ok", "conclusion": "success"}, + ] + } + net.runs[0]["conclusion"] = "failure" + fails = _fails(_checks()) + assert len(fails) == 3 and all(f.check.startswith("(e)") for f in fails) + assert "build mac (failure)" in fails[0].evidence + + +def test_e_cancelled_newest_run_is_skipped_but_shown(net): + net.runs.insert( + 0, + { + "databaseId": 9, + "status": "completed", + "conclusion": "cancelled", + "event": "push", + "headSha": "b" * 40, + "createdAt": "t", + }, + ) + results = [r for r in _checks() if r.check.startswith("(e)")] + assert all( + r.status == "PASS" + and "run 7" in r.evidence + and "completed/cancelled" in r.evidence + for r in results + ) + + +def test_e_in_progress_run_does_not_count_as_a_verdict(net): + net.runs = [ + { + "databaseId": 9, + "status": "in_progress", + "conclusion": None, + "event": "push", + "headSha": "b" * 40, + "createdAt": "t", + } + ] + assert all("no completed run" in r.evidence for r in _fails(_checks())) + + +def test_e_no_jobs_or_no_success(net): + net.jobs[7] = {"jobs": []} + assert "no jobs" in _fails(_checks())[0].evidence + net.jobs[7] = {"jobs": [{"name": "a", "conclusion": "skipped"}]} + assert "no job succeeded" in _fails(_checks())[0].evidence + + +def test_e_gh_error_fails_closed(net): + net.gh_error = "gh: command not found" + fails = _fails(_checks()) + assert fails and all( + "fail closed" in f.evidence for f in fails if f.check.startswith("(e)") + ) + + +# (f) +@pytest.mark.parametrize("version", [V, V + ".post1", V + ".dev0"]) +def test_f_real_workflows_derive_the_tag(net, version): + results = [r for r in _checks(version) if r.check.startswith("(f)")] + assert len(results) == 3 and all(r.status == "PASS" for r in results), results + assert any( + "image arcadedata/arcadedb:" + ("latest" if "dev" in version else V) + in r.evidence + for r in results + ) + + +def test_f_extract_step_run_finds_the_real_steps(): + text = (REPO / ".github/workflows/release-jvm.yml").read_text() + script = rp.extract_step_run(text, "Validate tag") + assert script and 'IMAGE_TAG="$BASE"' in script and not script.startswith(" ") + assert rp.extract_step_run(text, "No such step") is None + + +def test_f_missing_step_fails(net, tmp_path): + for rel in ( + ".github/workflows/release-jvm.yml", + ".github/workflows/release-native.yml", + ".github/workflows/test-native.yml", + ): + dst = tmp_path / rel + dst.parent.mkdir(parents=True, exist_ok=True) + text = ( + (REPO / rel) + .read_text() + .replace("- name: Validate tag", "- name: Check the tag") + ) + dst.write_text(text) + results = [r for r in _checks(root=tmp_path) if r.check.startswith("(f)")] + assert [r.status for r in results] == ["FAIL", "FAIL", "PASS"] + assert "has no step" in results[0].evidence + + +def test_f_wrong_derivation_fails(net, monkeypatch): + def lying_run(cmd, env=None, cwd=None): + if cmd[0] == "bash": + with open(env["GITHUB_OUTPUT"], "w") as f: + f.write( + "version=99.1.1\nbase-version=99.1.0\narcadedb-tag=latest\nfinal=false\nengine=99.0.0\n" + ) + return 0, "", "" + return net.run(cmd, env, cwd) + + monkeypatch.setattr(rp, "run", lying_run) + results = [r for r in _checks() if r.check.startswith("(f)")] + assert [r.status for r in results] == ["FAIL"] * 3 + assert "image tag 'latest', expected '99.1.1'" in results[0].evidence + assert "final 'false'" in results[1].evidence + assert "engine '99.0.0', expected '99.1.1'" in results[2].evidence + + +def test_f_step_that_exits_nonzero_fails(net, monkeypatch): + monkeypatch.setattr( + rp, + "run", + lambda cmd, env=None, cwd=None: ( + (1, "", "boom") if cmd[0] == "bash" else net.run(cmd, env, cwd) + ), + ) + assert all( + r.status == "FAIL" and "exited 1" in r.evidence + for r in _checks() + if r.check.startswith("(f)") + ) + + +def _as(monkeypatch, os_name): + # a stand-in for the os module of the script only: patching os.name itself would break pathlib on Windows + monkeypatch.setattr( + rp, "os", types.SimpleNamespace(name=os_name, environ=os.environ) + ) + + +def test_f_without_bash_is_a_skip(net, monkeypatch): + _as(monkeypatch, "posix") + monkeypatch.setattr(rp.shutil, "which", lambda name: None) + results = [r for r in _checks() if r.check.startswith("(f)")] + assert [r.status for r in results] == ["SKIP"] and "bash not found" in results[ + 0 + ].evidence + + +def test_f_on_windows_is_a_skip_and_runs_no_bash(net, monkeypatch): + _as(monkeypatch, "nt") + monkeypatch.setattr( + rp, + "run", + lambda cmd, env=None, cwd=None: ( + (_ for _ in ()).throw(AssertionError("bash must not run")) + if cmd[0] == "bash" + else net.run(cmd, env, cwd) + ), + ) + results = [r for r in _checks() if r.check.startswith("(f)")] + assert [r.status for r in results] == ["SKIP"] and "Windows" in results[0].evidence + assert not _fails(_checks()) + + +# (g) +def test_g_variable_false_or_unset_fails(net): + net.gh_variable = (0, "false\n", "") + assert "publishes no final wheel" in _only_fail(_checks(), "(g)").evidence + net.gh_variable = (1, "", "gh: Not Found (HTTP 404)") + assert "not set" in _only_fail(_checks(), "(g)").evidence + + +def test_g_unreadable_variable_is_a_skip(net): + net.gh_variable = (1, "", "gh: Must have admin rights (HTTP 403)") + results = [r for r in _checks() if r.check.startswith("(g)")] + assert results[0].status == "SKIP" and "403" in results[0].evidence + assert not _fails(_checks()) + + +# main +def test_exit_code_is_the_number_of_fails(net, monkeypatch, capsys): + net.http.clear() + monkeypatch.setattr(rp, "http_get", net.http_get) + code = rp.main([V, "--checkout", str(REPO)]) + out = capsys.readouterr().out + assert code == sum(1 for line in out.splitlines() if line.startswith("FAIL ")) + assert code >= 4 and f"{code} FAIL(s)" in out + assert "Ready" not in out + + +def test_main_ready_prints_the_go(net, capsys): + assert rp.main([V, "--checkout", str(REPO)]) == 0 + assert "Ready" in capsys.readouterr().out + + +def test_output_is_ascii(net, capsys): + rp.main([V, "--checkout", str(REPO)]) + capsys.readouterr().out.encode("ascii")