Repository navigation
task: agent ready prep #280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
823e526
9cefba6
3f8573f
67f65c0
0d3f956
ec65659
27d9109
781390f
4aed2ad
1c36162
3f6872c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Keep shell scripts LF on checkout so shellcheck and conda build scripts work | ||
| # the same on Windows and WSL checkouts with core.autocrlf enabled. | ||
| *.sh text eol=lf |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| name: Bug report | ||
| description: Report incorrect results, a crash, or a build failure | ||
| labels: ["bug"] | ||
| body: | ||
| - type: markdown | ||
| attributes: | ||
| value: | | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| For security vulnerabilities, do not open an issue — follow [SECURITY.md](https://github.com/IntelPython/mkl_umath/blob/main/SECURITY.md). | ||
|
|
||
| - type: textarea | ||
| id: description | ||
| attributes: | ||
| label: Description | ||
| description: What happened, and what did you expect instead? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: reproducer | ||
| attributes: | ||
| label: Reproducer | ||
| description: A minimal, self-contained snippet. Include the ufunc, input values, and dtype. | ||
| render: python | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: dropdown | ||
| id: install-source | ||
| attributes: | ||
| label: How was `mkl_umath` installed? | ||
| options: | ||
| - Intel conda channel (software.repos.intel.com) | ||
| - conda-forge | ||
| - pip, Intel index (software.repos.intel.com) | ||
| - pip / PyPI | ||
| - Built from source | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: versions | ||
| attributes: | ||
| label: Versions | ||
| description: | | ||
| Output of: | ||
| ``` | ||
| python -c "import mkl_umath, numpy; print(mkl_umath.__version__); print(numpy.__version__)" | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| ``` | ||
| Add the MKL version (`conda list mkl` or `pip list | grep -i mkl`), your OS, and your Python version too. | ||
|
Comment on lines
+44
to
+49
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The form invites build failures (line 2, and "Built from source" at line 36), but this required field needs
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added in 3f6872c: the Versions field asks for the compiler and version, the |
||
| If you built from source, or the build failed: the compiler and version (`icx --version` or `clang --version`), the `-Dmkl_threading` value, and the build log. Skip the `python -c` line if the import fails. | ||
| render: shell | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: notes | ||
| attributes: | ||
| label: Anything else | ||
| description: | | ||
| Optional. Whether NumPy patching was active (`mkl_umath.is_patched()`), whether the result differs from stock NumPy, or a non-default `MKL_NUM_THREADS`. | ||
| validations: | ||
| required: false | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| blank_issues_enabled: true | ||
| contact_links: | ||
| - name: Security vulnerability | ||
| url: https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html | ||
| about: Report security vulnerabilities through Intel's process, not a public issue. | ||
| - name: Question about usage | ||
| url: https://github.com/IntelPython/mkl_umath/blob/main/README.md | ||
| about: Check the README first, including the patching section. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| name: Feature request | ||
| description: Propose a new ufunc loop, dtype, or capability | ||
| labels: ["enhancement"] | ||
| body: | ||
| - type: textarea | ||
| id: problem | ||
| attributes: | ||
| label: What problem does this solve? | ||
| description: The use case, not the implementation. | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: proposal | ||
| attributes: | ||
| label: Proposal | ||
| description: | | ||
| What you would like `mkl_umath` to do. If it covers a NumPy ufunc, name it — NumPy's semantics are the contract for patched loops. | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: input | ||
| id: upstream | ||
| attributes: | ||
| label: Upstream equivalent | ||
| description: Link to the NumPy docs for the ufunc, if there is one. | ||
| validations: | ||
| required: false | ||
|
|
||
| - type: textarea | ||
| id: alternatives | ||
| attributes: | ||
| label: Alternatives considered | ||
| description: Optional. Workarounds you are using today. | ||
| validations: | ||
| required: false |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Description | ||
|
|
||
| <!-- What changed and why. Link any related issues. --> | ||
|
|
||
| ## Verification | ||
|
|
||
| <!-- The commands you ran, and the platform and versions you ran them on. --> | ||
|
|
||
| - Tests: <!-- e.g. `pytest mkl_umath/tests`, Python 3.12 / NumPy 2.x, Linux --> | ||
| - Lint: <!-- `pre-commit run --all-files` --> | ||
|
|
||
| ## Not verified | ||
|
|
||
| <!-- | ||
| Anything skipped or left to CI, and why. Examples: Windows, the Intel-channel | ||
| conda build, the benchmarks. Write "none" if you ran everything relevant. | ||
| --> | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Results match stock NumPy, or the difference is intentional and called out above. | ||
| - [ ] Behavior changes have tests in `mkl_umath/tests/`; bug fixes have a regression test. | ||
| - [ ] `CHANGELOG.md` updated under `## [dev]` with a `[gh-NNN](https://github.com/IntelPython/mkl_umath/pull/NNN)` link, or the change isn't user-visible. | ||
|
|
||
| <!-- See CONTRIBUTING.md for the build and test workflow, and AGENTS.md for the module map. --> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,109 +1,82 @@ | ||
| # AGENTS.md | ||
| # AGENTS.md — mkl_umath | ||
|
|
||
| Entry point for agent context in this repo. | ||
|
|
||
| ## What this repository is | ||
| `mkl_umath` exposes Intel® OneMKL-powered universal function loops for NumPy, originally part of Intel® Distribution for Python* and factored out per NEP-36 (Fair Play). | ||
|
|
||
| It provides: | ||
| - `mkl_umath._ufuncs` — OneMKL-backed NumPy ufunc loops | ||
| - `mkl_umath._patch_numpy` — runtime patching interface (`patch_numpy_umath` `restore_numpy_umath`, `is_patched()`) | ||
| - Performance-optimized math operations (sin, cos, exp, log, etc.) using Intel MKL VM | ||
| ## What this project is | ||
| `mkl_umath` provides NumPy ufunc loops backed by Intel® oneMKL Vector Math, and | ||
| can patch them into NumPy at runtime. It was factored out of Intel® Distribution | ||
| for Python* per NEP-36 (Fair Play). | ||
|
|
||
| ## Key components | ||
| - **Python interface:** `mkl_umath/__init__.py`, `_init_helper.py` | ||
| - **Core C implementation:** `mkl_umath/src/` (ufuncsmodule.c, mkl_umath_loops.c.src) | ||
| - **Cython patch layer:** `mkl_umath/src/_patch_numpy.pyx` | ||
| - **Code generation:** `generate_umath.py`, `generate_umath_doc.py` | ||
| - **Build system:** meson-python + Cython | ||
|
|
||
| ## Build dependencies | ||
| **Required:** | ||
| - Compiler toolchain: Intel `icx` or `clang` (with Intel-only flags gated when using clang) | ||
| - Intel® oneMKL (`mkl-devel`) | ||
| - meson-python, CMake, Ninja, Cython>=3.1.0, NumPy | ||
|
|
||
| **Build against an existing `mkl` installation:** | ||
|
|
||
| Install the build dependencies via Conda: | ||
| ```bash | ||
| conda install -c https://software.repos.intel.com/python/conda \ | ||
| mkl-devel dpcpp_linux-64 "cython>=3.1.0" meson-python cmake ninja numpy | ||
| ``` | ||
| or via pip: | ||
| ```bash | ||
| pip install mkl-devel "cython>=3.1.0" meson-python cmake ninja numpy | ||
| ``` | ||
| then build: | ||
| ```bash | ||
| CC=icx pip install --no-deps --no-build-isolation . # clang is also supported in CI | ||
| ``` | ||
|
|
||
| ## CI/CD | ||
| - **Platforms:** Linux, Windows | ||
|
jharlow-intel marked this conversation as resolved.
|
||
| - **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14, 3.15 | ||
| - **Workflows:** `.github/workflows/` | ||
| - `conda-package.yml` — main conda build/test pipeline | ||
| - `conda-package-cf.yml` — conda-forge-oriented build/test pipeline | ||
| - `build_pip.yml` — validates pip build with pre-release NumPy | ||
| - `build-with-clang.yml` — Intel clang compatibility check | ||
| - `build-with-standard-clang.yml` — standard clang compatibility check | ||
| - `openssf-scorecard.yml` — security scorecard | ||
|
|
||
| ## Distribution | ||
| - **Conda:** `https://software.repos.intel.com/python/conda` | ||
| - **PyPI:** `https://software.repos.intel.com/python/pypi` | ||
| - Requires Intel-optimized NumPy from Intel channels | ||
|
|
||
| ## Usage | ||
| ```python | ||
| import mkl_umath | ||
| mkl_umath.patch_numpy_umath() # Patch NumPy to use MKL loops | ||
| # ... perform NumPy operations (now accelerated) ... | ||
| mkl_umath.restore_numpy_umath() # Restore original NumPy loops | ||
| ``` | ||
|
|
||
| ## How to work in this repo | ||
| - **Performance:** Changes should maintain or improve MKL VM utilization | ||
| - **Compatibility:** Must work with upstream NumPy APIs (NEP-36 compliance) | ||
| - **Testing:** Add tests to `mkl_umath/tests/test_basic.py` | ||
| - **Build hygiene:** `meson.build` is the source of truth for build config — verify Linux + Windows | ||
| - **Docs:** Update docstrings via `ufunc_docstrings_numpy2.py` | ||
|
|
||
| ## Code structure | ||
| - **Generated code:** `*.src` files are templates (conv_template.py processes them) | ||
| - **Precision flags:** fp:precise, fimf-precision=high, fprotect-parens (non-negotiable) | ||
| - **Security:** Stack protection, FORTIFY_SOURCE, NX/DEP enforced in `meson.build` | ||
| - **Build options:** `opt_report` and `mkl_threading` are exposed via `meson.options` | ||
|
|
||
|
|
||
| ## Common pitfalls | ||
| - **NumPy source:** Requires Intel-optimized NumPy from Intel channel (`software.repos.intel.com/python/conda`). PyPI NumPy may cause runtime failures or incorrect results. | ||
| - **Precision flags:** `fp:precise`, `fimf-precision=high` enforce IEEE 754 compliance. Removing them breaks numerical correctness in scientific computing. | ||
| - **Patching order:** If using multiple Intel patches (e.g., `mkl_random` + `mkl_umath`), apply `mkl_umath` last. Verify with `is_patched()` after each. | ||
| - **Compiler/toolchain:** `icx` and `clang` are both supported; when using clang, keep Intel-only flags behind compiler guards. | ||
| - **Build validation:** | ||
| - After setup: `which ${CC:-icx}` → should resolve to the intended compiler toolchain | ||
| - Check: `python -c "import numpy; print(numpy.__version__)"` → confirm NumPy is available | ||
|
|
||
| ## Notes | ||
| - `_vendored/` contains vendored NumPy code generation utilities | ||
| - Version in `mkl_umath/_version.py` (read dynamically by `meson.build`) | ||
| - Patching is runtime-only; no NumPy source modification | ||
| - **Package and public API:** `mkl_umath/`, `mkl_umath/__init__.py` | ||
| - **Ufunc extension:** `mkl_umath/src/ufuncsmodule.c`, plus `__umath_generated.c` | ||
| from `mkl_umath/generate_umath.py` and `_umath_doc_generated.h` from | ||
| `mkl_umath/ufunc_docstrings_numpy2.py` | ||
| - **Loop templates:** `mkl_umath/src/mkl_umath_loops.{c,h}.src` | ||
| - **Patching:** `mkl_umath/src/_patch_numpy.pyx`; persistent and one-shot | ||
| patching in `mkl_umath/patch.py`, `mkl_umath/with_patch.py`, | ||
| `mkl_umath/_patch_startup.py`, and the `mkl_umath/__main__.py` CLI | ||
| - **Tests:** `mkl_umath/tests/` | ||
| - **Vendored helpers:** `_vendored/` | ||
| - **Packaging:** `conda-recipe/`, `conda-recipe-cf/` | ||
| - **Benchmarks:** `benchmarks/` | ||
|
|
||
| ## Build/runtime basics | ||
| - Build system: `pyproject.toml` + `meson.build`, with options in `meson.options` | ||
| - Build deps: `mkl-devel`, `numpy`, `meson-python`, `cmake`, `ninja`, `cython`, | ||
| and a C compiler (CI uses `icx` and `clang`) | ||
| - Runtime deps: `numpy`, NumPy 2 only (see `pyproject.toml`); the conda recipes | ||
| add the MKL and compiler runtimes | ||
| - Platforms: Linux and Windows; `.github/workflows/` is canonical for the matrix | ||
| - Setup, checks, and style: `CONTRIBUTING.md` | ||
| - Single test: `pytest mkl_umath/tests/<file>.py::<test>`, from a checkout with the | ||
| editable install in `CONTRIBUTING.md` | ||
| - Single-file lint: `pre-commit run --files <path>`, from a branch (see | ||
| `CONTRIBUTING.md`) | ||
|
|
||
| ## Development guardrails | ||
| - Preserve NumPy ufunc behavior; patched loops stand in for NumPy's own. | ||
| - Edit the `*.src` templates, `generate_umath.py`, and | ||
| `ufunc_docstrings_numpy2.py`, not generated C or headers. | ||
| - Keep the floating-point precision flags in `meson.build`: `-fno-fast-math`, | ||
| `-fimf-precision=high`, `-fprotect-parens` (GCC syntax) and `/fp:precise`, | ||
| `/Qimf-precision=high`, `/Qprotect-parens` (MSVC syntax). Both recipes' | ||
| `conda_build_config.yaml` also set `/fp:precise` on Windows. Intel-only flags | ||
| stay behind the compiler checks in `meson.build`. | ||
| - Keep the hardening flags in `meson.build`: `-fstack-protector-all`, | ||
| `-D_FORTIFY_SOURCE=2`, `-z relro`/`now`/`noexecstack`, `/GS`, `/DynamicBase`, | ||
| and `/NXCompat`. | ||
| - Performance changes must not compromise numerical correctness. | ||
| - Avoid hardcoded ISA assumptions beyond what the build configuration sets. | ||
| - `meson.build` changes must keep both the GCC-syntax (Linux) and MSVC-syntax | ||
| (Windows) branches working; CI covers both. | ||
| - Preserve the patching API (`patch_numpy_umath()`, `restore_numpy_umath()`, | ||
| `is_patched()`, and the `mkl_umath()` context manager) unless asked to change | ||
| it. The deprecated `use_in_numpy()` and `restore()` aliases stay until a | ||
| release removes them. | ||
| - Keep patching reversible, with `is_patched()` reporting the truth. | ||
| - Keep both extensions free-threading compatible. | ||
| - Pair behavior changes with tests and keep diffs minimal. | ||
| - Avoid hardcoding mutable versions/matrices/channels in docs, and never invent | ||
| versions, compilers, or CI matrix values; take them from the files below. | ||
| - Never put secrets or credentials in code or docs. | ||
|
|
||
| ## Where truth lives | ||
| - Build/config: `pyproject.toml`, `meson.build`, `meson.options` | ||
| - Dependencies: `pyproject.toml`, `conda-recipe*/meta.yaml` | ||
| - CI/workflows: `.github/workflows/*.yml` | ||
| - Public API: `mkl_umath/__init__.py`, `mkl_umath/src/_patch_numpy.pyx` | ||
| - Tests: `mkl_umath/tests/` | ||
|
|
||
| For behavior policy, see `.github/copilot-instructions.md`. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Following up on the earlier thread on
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in 3f6872c: all four are now in the root |
||
|
|
||
| ## Directory map | ||
| Below directories have local `AGENTS.md` for deeper context: | ||
| - `.github/AGENTS.md` — CI/CD workflows and automation | ||
| - `mkl_umath/AGENTS.md` — Python API and code generation | ||
| - `mkl_umath/src/AGENTS.md` — C/Cython implementation layer | ||
| - `mkl_umath/tests/AGENTS.md` — unit tests and validation | ||
| - `conda-recipe/AGENTS.md` — Intel channel conda packaging | ||
| - `conda-recipe-cf/AGENTS.md` — conda-forge compatible recipe | ||
| - `_vendored/AGENTS.md` — vendored NumPy utilities | ||
|
|
||
| --- | ||
|
|
||
| For broader IntelPython ecosystem context, see: | ||
| - `dpnp` (Data Parallel NumPy) | ||
| - `mkl_random` (MKL-based random number generation) | ||
| - `numba-dpex` (Numba + SYCL) | ||
| Use nearest local `AGENTS.md` when present: | ||
| - `.github/AGENTS.md` — CI workflows and automation policy | ||
| - `mkl_umath/AGENTS.md` — package modules, API, and code generation | ||
| - `mkl_umath/src/AGENTS.md` — loop templates and the two extensions | ||
| - `mkl_umath/tests/AGENTS.md` — test scope and conventions | ||
| - `conda-recipe/AGENTS.md` — Intel-channel conda packaging | ||
| - `conda-recipe-cf/AGENTS.md` — conda-forge recipe | ||
| - `_vendored/AGENTS.md` — vendored NumPy template tooling | ||
| - `benchmarks/AGENTS.md` — ASV performance suite | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Policy section could list the conventions for editing workflows, which every workflow here already follows and zizmor checks: pin each action to a full commit SHA with a
# vX.Y.Zcomment, set least-privilegepermissions, and usepersist-credentials: falseon checkouts.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in 3f6872c.