Repository navigation
Drop the conda package cache from the conda CI workflows - #262
Open
antonwolfy wants to merge 1 commit into
Open
antonwolfy wants to merge 1 commit into
antonwolfy wants to merge 1 commit into
Conversation
The actions/cache steps in conda-build-test.yml cost 12-17% of the runner time of a conda-package or conda-package-cf run without making any step faster: - Build jobs never restored a cache: each run saves about sixty ~1 GB entries, so the previous run's entries are evicted before the next run starts. - Test jobs only restored the build job's package directory through the restore-keys prefix, and the Intel and conda-forge workflows shared the same keys, so about half of them got the other channel's packages and downloaded everything anyway. - The install step took the same time whether its 120-190 MB of packages came from the cache or from the channels. Remove both cache steps, the lockfile-only "Collect dependencies" step, CONDA_PKGS_DIRS and the pkgs-dir input, and replace the cache key note in .github/AGENTS.md.
antonwolfy
requested review from
jharlow-intel,
ndgrigorian,
vlad-perevezentsev and
xaleryb
as code owners
October 28, 2026 11:04
This was referenced Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
Cache conda packagessteps inconda-build-test.ymlmake the conda workflows slower and save nothing. In today's runs they took 11-13 of ~97 runner-minutes perconda-package.ymlrun and 33 of ~197 perconda-package-cf.ymlrun (12-17%), mostly on macOS. No step got faster in return.Measured on 158 successful jobs from 4 runs on 2026-10-28: master 40669276700 / 40669276773 and PR 40669327541 / 40669327527.
Install mkl-servicewhen its packages were in the cacheInstall mkl-servicewhen it downloaded 120-190 MBWhy it doesn't help:
.condaarchives and their unpacked copies. Each run saves about sixty of them. The repo's cache store held 20 entries (20.6 GB), only macOS ones from the latest run, so the build key (meta.yamlhash) never gets an exact hit.restore-keysprefix picks up the build job's package directory saved minutes earlier, and the<os>-conda-0-fallback gave 3 of the 8 macOS test jobs the Python 3.13 entry.conda-package.ymlandconda-package-cf.ymluse the same keys. 28 saves failed on the collision, and about half the test jobs restored the other channel's packages and downloaded everything anyway.Changes
conda-build-test.yml: remove bothCache conda packagessteps, theCollect dependenciesstep (its dry-run lockfile was only used for the cache key;Install mkl-serviceprints the same plan), theCONDA_PKGS_DIRSenv and thepkgs-dirinputconda-package.yml,conda-package-cf.yml: stop passingpkgs-dir.github/AGENTS.md: replace the cache-key note with why the conda jobs don't cache packagesJob and check names are unchanged. The existing cache entries are deleted automatically after 7 days without use.
Test plan
pre-commit(including actionlint) passes on the changed fileszizmor --min-severity medium .github/reports no findingsConda package using IntelandConda package using conda-forgeare green on this PR (all 80 conda jobs;build-with-clangfails on theapt-keyissue already on master, fixed by Replace apt-key in IntelLLVM clang workflow #261)