From 29e711237706d7c70127d7fd1c38204f3214935a Mon Sep 17 00:00:00 2001 From: Anton Volkov Date: Wed, 28 Oct 2026 12:33:57 +0100 Subject: [PATCH] Build the conda-forge workflow like the conda-forge feedstock Make conda-package-cf.yml build and test conda-recipe-cf the way conda-forge/mkl-service-feedstock does, so problems show up here before a feedstock PR: - Pin gcc 15 on Linux, clang 21 on macOS and mkl_devel 2026 (not on macOS) in conda_build_config.yaml, as conda-forge pinning does. The compilers were unpinned and resolved to gcc 16.2 and clang 23.1, so the Linux package required libgcc >=16 instead of >=15. - Drop Python 3.10, which conda-forge no longer builds, through a new python-versions input of conda-build-test.yml. 3.15t stays, as an early check before conda-forge starts its migration. - Build on windows-2022 for Visual Studio 2022, as the feedstock does. windows-latest only ships Visual Studio 2026. - Pass -v to pip in build.sh and bld.bat, so the meson output shows in the logs, as in the feedstock. Replace the test job's own install-and-pytest steps with "conda build --test" on the built package, in both workflows. CI then runs the test section of the recipe, as the feedstock does: pip check, the import test and pytest for conda-recipe-cf, the import test and pytest for conda-recipe. The test environment's Python comes from the package's python_abi dependency, so the free-threaded builds still get a free-threaded interpreter. The build job keeps --no-test, so the tests run once, on a fresh runner. Move the matrix values used in run steps into env, since zizmor flags them once the matrix depends on an input, and update the AGENTS.md files. --- .github/AGENTS.md | 6 ++- .github/workflows/conda-build-test.yml | 63 +++++++++---------------- .github/workflows/conda-package-cf.yml | 7 ++- AGENTS.md | 4 +- conda-recipe-cf/bld.bat | 2 +- conda-recipe-cf/build.sh | 2 +- conda-recipe-cf/conda_build_config.yaml | 6 +++ conda-recipe/AGENTS.md | 8 ++-- mkl/tests/AGENTS.md | 5 +- 9 files changed, 48 insertions(+), 55 deletions(-) diff --git a/.github/AGENTS.md b/.github/AGENTS.md index a86c28bf..37ebce98 100644 --- a/.github/AGENTS.md +++ b/.github/AGENTS.md @@ -3,10 +3,10 @@ CI/CD workflows, automation, security scanning, and package distribution. ## Workflows -All build/test workflows cover Python 3.10-3.15 plus free-threaded 3.14t and 3.15t. +All build/test workflows cover Python 3.10-3.15 plus free-threaded 3.14t and 3.15t, except `conda-package-cf.yml`, which starts at 3.11 like the feedstock. - **conda-package.yml** — main build/test pipeline (Linux/Windows) -- **conda-package-cf.yml** — build/test using only conda-forge channel (Linux/Windows) +- **conda-package-cf.yml** — build/test using only conda-forge channel (Linux/Windows/macOS), set up like `conda-forge/mkl-service-feedstock`: Windows uses `windows-2022` (Visual Studio 2022) - **build-with-clang.yml** — Linux Clang compiler compatibility validation - **build-with-standard-clang.yml** — standard Clang compiler compatibility validation - **build_pip.yml** — validates editable build @@ -22,6 +22,8 @@ All build/test workflows cover Python 3.10-3.15 plus free-threaded 3.14t and 3.1 - `include` rows create new matrix combinations and inherit nothing, so `experimental` must be repeated wherever the job reads them - Artifact naming: `$PACKAGE_NAME $OS Python $VERSION`, where `$VERSION` is `python_tag` (`3.14`, `3.14t`, `3.15`, or `3.15t`) when set, otherwise `python` - Channels: `conda-forge`, Intel channel +- Build jobs run `conda build --no-test`; test jobs run `conda build --test` on the built package on a fresh runner, so CI runs the test section of the recipe, as the feedstock does +- `conda-package-cf.yml` and `conda-recipe-cf/` follow `conda-forge/mkl-service-feedstock`: when the feedstock is re-rendered with new pins (compiler versions, `mkl_devel`, Python versions), update `conda-recipe-cf/conda_build_config.yaml` and the `python-versions` input to match ## Security - OpenSSF Scorecard runs automatically diff --git a/.github/workflows/conda-build-test.yml b/.github/workflows/conda-build-test.yml index 216d004c..cd2be24d 100644 --- a/.github/workflows/conda-build-test.yml +++ b/.github/workflows/conda-build-test.yml @@ -24,16 +24,17 @@ on: required: false default: false type: boolean + python-versions: + description: JSON list of Python versions to build and test in addition to 3.14 and 3.15, which always get both the GIL and free-threaded builds + required: false + default: '["3.10", "3.11", "3.12", "3.13"]' + type: string permissions: contents: read env: - module-name: mkl package-name: mkl-service - ver-json-path: 'version.json' - ver-script-part1: "import json; f = open('version.json', 'r'); j = json.load(f); f.close(); " - ver-script-part2: "d = j['mkl-service'][0]; print('='.join((d[s] for s in ('version', 'build'))))" jobs: build: @@ -42,7 +43,7 @@ jobs: strategy: fail-fast: false matrix: - python: ['3.10', '3.11', '3.12', '3.13'] + python: ${{ fromJSON(inputs.python-versions) }} include: - python: '3.14' python_spec: '3.14.* *_cp314' @@ -98,10 +99,11 @@ jobs: env: CHANNELS_LIST: ${{ inputs.channels-list }} RECIPE_DIR: ${{ inputs.recipe-dir }} + PYTHON_SPEC: ${{ matrix.python_spec || matrix.python }} run: | # CHANNELS_LIST intentionally word-splits into multiple -c args # shellcheck disable=SC2086 - conda build --no-test --python "${{ matrix.python_spec || matrix.python }}" ${CHANNELS_LIST} "${RECIPE_DIR}" + conda build --no-test --python "${PYTHON_SPEC}" ${CHANNELS_LIST} "${RECIPE_DIR}" - name: Upload artifact uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 @@ -123,7 +125,7 @@ jobs: strategy: fail-fast: false matrix: - python: ['3.10', '3.11', '3.12', '3.13'] + python: ${{ fromJSON(inputs.python-versions) }} include: - python: '3.14' python_spec: '3.14.* *_cp314' @@ -146,54 +148,31 @@ jobs: shell: bash -el {0} env: - test-env-name: 'test' - channel-path: './channel' - pkg-path-in-channel: './channel/${{ inputs.conda-subdir }}' + pkg-dir: './pkgs' steps: - name: Download artifact uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 with: name: ${{ env.package-name }} ${{ runner.os }} Python ${{ matrix.python_tag || matrix.python }} - path: ${{ env.pkg-path-in-channel }} + path: ${{ env.pkg-dir }} - name: Setup miniconda uses: conda-incubator/setup-miniconda@be893c923ea9cf1cf7cd510fbdde27c7e18cbdcb # v4.1.0 with: miniforge-version: latest channels: conda-forge - activate-environment: ${{ env.test-env-name }} - python-version: ${{ matrix.python }} - - - name: Install conda-index - run: conda install -n base -y conda-index - - - name: Create conda channel - run: | - conda run -n base python -m conda_index ${{ env.channel-path }} - - - name: Test conda channel - run: | - conda search ${{ env.package-name }} -c ${{ env.channel-path }} --override-channels --info --json > ${{ env.ver-json-path }} - cat ${{ env.ver-json-path }} - - - name: Get package version - run: | - PACKAGE_VERSION=$(python -c "${{ env.ver-script-part1 }} ${{ env.ver-script-part2 }}") - export PACKAGE_VERSION + activate-environment: '' - echo "PACKAGE_VERSION=${PACKAGE_VERSION}" - echo "PACKAGE_VERSION=$PACKAGE_VERSION" >> "$GITHUB_ENV" + - name: Install conda-build + run: conda install -n base -y conda-build - - name: Install mkl-service - run: | - conda install ${{ env.package-name }}=${{ env.PACKAGE_VERSION }} pytest "python=${{ matrix.python_spec || matrix.python }}" ${{ env.CHANNELS }} + - name: Test conda package env: - CHANNELS: '-c ${{ env.channel-path }} ${{ inputs.channels-list }}' - - - name: List installed packages - run: conda list - - - name: Run tests + CHANNELS_LIST: ${{ inputs.channels-list }} run: | - pytest -vv --pyargs ${{ env.module-name }} + # Runs the test section of the recipe embedded in the package, in a + # new environment whose Python comes from the package's python_abi + # CHANNELS_LIST intentionally word-splits into multiple -c args + # shellcheck disable=SC2086 + conda build --test "${{ env.pkg-dir }}"/${{ env.package-name }}-*.conda ${CHANNELS_LIST} diff --git a/.github/workflows/conda-package-cf.yml b/.github/workflows/conda-package-cf.yml index 2eb62935..4605525a 100644 --- a/.github/workflows/conda-package-cf.yml +++ b/.github/workflows/conda-package-cf.yml @@ -13,6 +13,8 @@ concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true +# Build like conda-forge/mkl-service-feedstock: same Python versions and +# Visual Studio 2022 on Windows jobs: linux: name: linux @@ -22,15 +24,17 @@ jobs: conda-subdir: linux-64 channels-list: -c conda-forge --override-channels recipe-dir: conda-recipe-cf + python-versions: '["3.11", "3.12", "3.13"]' windows: name: windows uses: ./.github/workflows/conda-build-test.yml with: - runs-on: windows-latest + runs-on: windows-2022 conda-subdir: win-64 channels-list: -c conda-forge --override-channels recipe-dir: conda-recipe-cf + python-versions: '["3.11", "3.12", "3.13"]' osx: name: osx @@ -40,3 +44,4 @@ jobs: conda-subdir: osx-64 channels-list: -c conda-forge --override-channels recipe-dir: conda-recipe-cf + python-versions: '["3.11", "3.12", "3.13"]' diff --git a/AGENTS.md b/AGENTS.md index c9d64151..d6d18bf4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,12 +48,12 @@ python -m pip install --no-deps --no-build-isolation . ## CI/CD - **Platforms in CI workflows:** Linux, Windows, macOS (conda-forge workflow only) -- **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15, and free-threaded 3.14t and 3.15t +- **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15, and free-threaded 3.14t and 3.15t (the conda-forge workflow starts at 3.11, like the feedstock) - **Python 3.14 ABI selection:** conda jobs must request an explicit ABI, `3.14.* *_cp314` (GIL) or `3.14.* *_cp314t` (free-threaded). Once the recipes stopped constraining `python-gil`, a bare `--python 3.14` was observed to resolve to the free-threaded build, so it must not be relied on. - **Python 3.15 ABI selection:** same as 3.14; conda jobs request `3.15.* *_cp315` (GIL) or `3.15.* *_cp315t` (free-threaded) explicitly. - **Workflows:** `.github/workflows/` - `conda-package.yml` — main conda build/test pipeline - - `conda-package-cf.yml` — conda build/test using only conda-forge channel + - `conda-package-cf.yml` — conda build/test using only conda-forge channel, set up like `conda-forge/mkl-service-feedstock` - `build-with-clang.yml` — Linux Clang compatibility - `build-with-standard-clang.yml` — standard Clang compiler compatibility validation - `build_pip.yml` — validates editable build diff --git a/conda-recipe-cf/bld.bat b/conda-recipe-cf/bld.bat index 79c126ea..8f81fe26 100644 --- a/conda-recipe-cf/bld.bat +++ b/conda-recipe-cf/bld.bat @@ -1,2 +1,2 @@ -%PYTHON% -m pip install --no-deps --no-build-isolation . +%PYTHON% -m pip install --no-deps --no-build-isolation -v . if errorlevel 1 exit 1 diff --git a/conda-recipe-cf/build.sh b/conda-recipe-cf/build.sh index 5f4b49d6..a39dcbfa 100644 --- a/conda-recipe-cf/build.sh +++ b/conda-recipe-cf/build.sh @@ -1,2 +1,2 @@ #!/bin/bash -ex -$PYTHON -m pip install --no-deps --no-build-isolation . +$PYTHON -m pip install --no-deps --no-build-isolation -v . diff --git a/conda-recipe-cf/conda_build_config.yaml b/conda-recipe-cf/conda_build_config.yaml index 619c3761..b0c71561 100644 --- a/conda-recipe-cf/conda_build_config.yaml +++ b/conda-recipe-cf/conda_build_config.yaml @@ -1,11 +1,15 @@ c_compiler: # [linux] - gcc # [linux] +c_compiler_version: # [linux] + - '15' # [linux] c_stdlib: # [linux] - sysroot # [linux] c_stdlib_version: # [linux] - '2.17' # [linux] c_compiler: # [osx] - clang # [osx] +c_compiler_version: # [osx] + - '21' # [osx] c_stdlib: # [osx] - macosx_deployment_target # [osx] c_stdlib_version: # [osx] @@ -16,3 +20,5 @@ c_stdlib: # [win] - vs # [win] c_compiler: # [win] - vs2022 # [win] +mkl_devel: # [not osx] + - '2026' # [not osx] diff --git a/conda-recipe/AGENTS.md b/conda-recipe/AGENTS.md index b9994b9d..0eeb6ae6 100644 --- a/conda-recipe/AGENTS.md +++ b/conda-recipe/AGENTS.md @@ -10,8 +10,8 @@ Conda package build recipe for conda-forge and Intel channel distribution. ## Build configuration - **Channels:** `conda-forge`, Intel channel -- **Python versions in CI build matrix:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15, and free-threaded 3.14t and 3.15t -- **Compilers:** pinned via `conda_build_config.yaml` (GCC/GXX on Linux, VS2022 on Windows) +- **Python versions in CI build matrix:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15, and free-threaded 3.14t and 3.15t (3.11+ for `conda-recipe-cf/`) +- **Compilers:** pinned via `conda_build_config.yaml` (GCC/GXX on Linux, VS2022 on Windows); `conda-recipe-cf/` also pins gcc 15, clang 21 and `mkl_devel` 2026, as the feedstock's conda-forge pinning does - **Dependencies:** `mkl` (runtime), `mkl-devel` (build), `cython >=3.1.0` ## Build outputs @@ -21,12 +21,12 @@ Conda package build recipe for conda-forge and Intel channel distribution. ## CI usage - Built in `.github/workflows/conda-package.yml` - Artifacts uploaded per Python version and platform -- Test stage uses built artifacts from a temporary local channel +- Test stage runs `conda build --test` on the built artifact, so the `test:` section of the recipe is what CI runs ## Platform specifics - **Linux:** RTLD_GLOBAL handling in runtime init path - **Windows:** MKL library path configuration -- **macOS:** recipe scripts exist, but current GitHub Actions matrix does not build/test macOS +- **macOS:** built and tested only from `conda-recipe-cf/` in `conda-package-cf.yml` ## Maintenance - Keep recipe metadata in sync with workflow matrix and package metadata diff --git a/mkl/tests/AGENTS.md b/mkl/tests/AGENTS.md index a1d7fe96..117655c8 100644 --- a/mkl/tests/AGENTS.md +++ b/mkl/tests/AGENTS.md @@ -25,9 +25,10 @@ pytest mkl/tests/ ``` ## CI integration -- Tests run in `conda-package.yml` workflow +- Tests run in the `conda-package.yml` and `conda-package-cf.yml` workflows, through `conda build --test`, which runs the `test:` section of `conda-recipe/meta.yaml` or `conda-recipe-cf/meta.yaml` +- Test-only dependencies go in `test: requires:` of both recipes; CI installs nothing else - Separate test jobs per Python version and CI platform -- CI coverage: Linux + Windows +- CI coverage: Linux + Windows, plus macOS in `conda-package-cf.yml` ## Adding tests - New API functions → add to `test_mkl_service.py` with validation