Skip to content

Build and test conda-recipe-cf for Intel Python CI - #204

Open
jharlow-intel wants to merge 3 commits into
masterfrom
ci/intel-python-ci
Open

jharlow-intel wants to merge 3 commits into
masterfrom
ci/intel-python-ci

Conversation

@jharlow-intel

@jharlow-intel jharlow-intel commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Adds what Intel Python CI (gh-ci) needs from mkl_random: a ci/ directory and a new workflow, .github/workflows/intel-ci.yml. The workflow builds conda-recipe-cf for every platform and Python version the recipe declares, and tests each package in a clean conda-forge environment with ci/test.py. Intel's Jenkins then waits for that run, tests each package again on its own machines, scans it, and for a release tag publishes the packages it tested. Jenkins isn't switched on for mkl_random yet, so until it is, this only adds a GitHub Actions workflow.

  • ci/matrix.py, ci/build.py, ci/hooks.py: the shared gh-ci scripts, standard library only. Here they look for conda-recipe-cf/ first and build Linux and Windows on GitHub-hosted runners. hooks.py installs the test requirements from ci/config.yml, the way Jenkins does.
  • ci/test.py: runs pytest --pyargs mkl_random against the installed package and writes a JUnit report.
  • ci/config.yml: the workflow, the artifact names (mkl_random {platform} Python {python}, as conda-package-cf.yml names them) and the test requirements, pytest.
  • conda-recipe-cf/conda_build_config.yaml: now lists the Python versions, the ones conda-package-cf.yml builds today, for ci/matrix.py, and numpy: '2'. Without a NumPy variant, conda-build falls back to NumPy 1.26, which has no builds for Python 3.13 and newer. conda-package-cf.yml passes --python and --numpy itself, so it is unaffected.
  • ci/README.md and AGENTS.md notes.

A pull request builds the two newest regular Python versions on Linux and Windows; pushes to master and tags build them all.

Miniforge is pinned to 26.7.2-0. Miniforge 26.9.4-0, released on 22 October, makes setup-miniconda's conda init fail on Windows runners, and the existing conda-package*.yml workflows, which install latest, hit it too on this pull request.

Overlap with #202. It also sets numpy: '2' in conda_build_config.yaml and drops Python 3.10, so whichever lands second needs a small rebase, and if 3.10 goes it should leave the python: list here too. conda-package-cf.yml and intel-ci.yml now both build conda-recipe-cf; retiring one can be a follow-up.

Verification

  • Tests: this pull request's Intel Python CI run built Linux and Windows for Python 3.14 and 3.15 and ran ci/test.py against each package: 380 passed on Linux 3.14 and on Windows 3.15.
  • Lint: pre-commit run on every changed file, all hooks passing, and zizmor on intel-ci.yml, no findings.
  • python ci/matrix.py lists 14 builds, Linux and Windows for 3.10 to 3.15 and 3.14t; with --event pull_request it lists 3.14 and 3.15 on both.
  • ci/config.yml loads with gh-ci's own parser.

Not verified

  • The builds a tag or a push to the default branch adds, Python 3.10 to 3.13 and 3.14t; they use the same jobs.
  • The Jenkins side, which isn't enabled for mkl_random yet.

Adds what Intel Python CI (gh-ci) needs from this repository: ci/ with the
shared matrix, build and test-hook scripts, ci/test.py and ci/config.yml, and
.github/workflows/intel-ci.yml, which builds conda-recipe-cf for every platform
and Python version the recipe declares and tests each package in a clean
conda-forge environment, as Jenkins later does.

conda-recipe-cf/conda_build_config.yaml now declares the Python versions
conda-package-cf.yml builds, for ci/matrix.py, and NumPy 2: without a numpy
variant conda-build falls back to NumPy 1.26, which has no builds for Python
3.13 and newer. conda-package-cf.yml passes --python and --numpy itself, so it
is unaffected.
@jharlow-intel
jharlow-intel marked this pull request as ready for review October 23, 2026 20:42
@antonwolfy antonwolfy added this to the 1.7.0 release milestone Oct 26, 2026

concurrency:
group: intel-ci-${{ github.ref }}
cancel-in-progress: true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With cancel-in-progress: true keyed on github.ref, two quick merges cancel the first commit's whole run. Jenkins, which waits for that run, gets no packages for it.

Suggested change
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Comment on lines +12 to +13
numpy:
- '2'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That only will work when ##202 is merged

@@ -1,3 +1,16 @@
# The Python versions to build, one per line and without selectors: ci/matrix.py
# reads them for .github/workflows/intel-ci.yml. NumPy 2, as in conda-forge's
# global pinning; conda-package-cf.yml passes its own with --python and --numpy.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--numpy is dropping in #202

Comment thread ci/hooks.py
"--artifacts", required=True, type=Path, help="the downloaded artifact"
)
parser.add_argument(
"--work", required=True, type=Path, help="an empty scratch directory"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hooks.py doesn't check that --work is empty, and may install conda-index into the user's base environment.


- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any issue to use the newer python?

activate-environment: ""
conda-remove-defaults: "true"

- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we can drop setup-python and run with shell: bash -el {0} with base activated.

bash is available on Windows runners too, and setup-miniconda supports this shell there. The step bodies become plain python ci/build.py … and python ci/hooks.py …. The $CONDA path code and hooks.py's re-run of itself would then only matter for local use.

@@ -0,0 +1,135 @@
# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same PR is open in three repos at once.
Have you considered moving common things (scripts, workflow) to some common repo like https://github.com/IntelPython/devops-tools ?
That would help to reduce the duplication.

A pull request now builds and tests the oldest and the newest regular Python
versions the recipe declares, instead of the two newest, so the oldest supported
Python is tested before a change merges rather than first on a release tag.
Intel's Jenkins tests the same two.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants