Faster CI: Windows builds on D:, Defender and Spotlight off, parallel builds by Tobias-Fischer · Pull Request #154 · RoboStack/robostack

Tobias-Fischer · 2026-10-24T04:58:03Z

Faster CI jobs on all platforms, from measurements on this PR (comments below).

Changes

	PR builds (testpr)	build branches
Windows: builds, rattler's package cache and TEMP on D: (about 3× the IOPS of C: on GitHub's runners)	✅	✅ `build_win.bat`
Windows: Defender real-time scan off	✅	✅ `build_win.bat`
macOS: Spotlight off (`mdutil -a -i off`)	✅	✅ `build_unix.sh`
Parallel builds: rattler-build 0.77 `--parallel-builds 3:<cores>` (jobserver); Windows one at a time	✅ one `rs build` run	✅ one `build-ci` run per job, see below
PR build estimate from the measured build times (`ci_estimate.py`)	✅	–

On the build branches, each job's recipes now build in one rattler-build run with --log-style json:

build-ci prints each line as HH:MM:SS [recipe] message, since parallel output interleaves.
It records each package's time (first to last line of its build), scaled so the times add up to the builds' wall time.
Compatibility recipes (ros-<distro>-*) keep the constant ALIAS_SECONDS.

Measurements

Each run rebuilt the same lyrical packages (temporary commits, dropped before merging). The times are the "Build recipes" step.

Windows, win-64, 6 packages:

run	setup	build step
C	C:, Defender off, 2 parallel builds	1216 s
D	D:, Defender off, 2 parallel builds	305 s (−75%)

| G | D:, Defender off, 1 build at a time (final) | 282 s |
| I | D:, Defender on, 1 build at a time | 441 s |

Turning Defender off still saves ~36% on D: (run I against G). Two parallel builds don't help on Windows (D against G). The Access is denied retries remain (MSVC's mspdbsrv.exe, Windows: test cleanup fails with Access is denied (os error 5) when a test spawns a lingering child process (MSVC mspdbsrv.exe) prefix-dev/rattler-build#2657) but take ~1 s.
windows-2022 has no Dev Drives, and a plain ReFS volume broke rattler-build's reindex (run E).

Parallel builds (run C against serial run B, normalised by the recipes' measured times):

platform	change
linux-64	−18%
linux-aarch64	−41%
osx-arm64	−25%
osx-64	−7% (within its noise)
win-64 (on C:)	+9%

Before and after, pre-PR setup (run F) against this PR's setup (mean of runs C, D, E, E′):

platform	before	after	change
linux-64	337 s	271 s	−20%
linux-aarch64	263 s	195 s	−26%
osx-arm64	435 s	347 s	−20%
osx-64	1051 s	759 s	−28% (noisy)
win-64 (run D)	1615 s	305 s	−81%

Repeatability: identical runs vary by ±8% on linux-64 and ±2% on linux-aarch64 (one outlier at +16%). osx-arm64 took 333–360 s in four runs but 485 s in a fifth, and osx-64's Intel runners vary by about a third. So the macOS numbers are rough.

Spotlight: osx-arm64 took 414 s with Spotlight on (run H), against 333–485 s with it off: unproven. It stays off as a one-line change that other projects report helping (PennyLaneAI/catalyst, CapSoftware/Cap).

Before merging

The branch now holds only the final changes (nine commits on main); the temporary test commits are gone.
PR CI never runs .scripts/build_*.sh/.bat, so their changes are first exercised by the build-branch runs after the merge. Watch the first Windows jobs.
This changes .github/, .scripts/ and tools/: merging regenerates every distribution's build branches.

🤖 Generated with Claude Code

github-actions · 2026-10-24T05:04:34Z

ABI check (linux-64)

lyrical linux-64: 10 can be bumped on their own

10 packages can be bumped on their own (compatible: 7, no libraries: 3)

package	released → PR	verdict	details
ackermann-steering-controller	6.9.0 (build 30) → 6.9.0	compatible: no ABI change	headers unchanged
autoware-map-height-fitter	1.7.0 (build 30) → 1.7.0	compatible: no ABI change	headers unchanged
autoware-planning-factor-interface	1.7.0 (build 30) → 1.7.0	compatible: no ABI change	headers unchanged
grid-map-rviz-plugin	2.4.0 (build 30) → 2.4.0	compatible: no ABI change	headers unchanged
plansys2-executor	3.1.0 (build 30) → 3.1.0	compatible: no ABI change	headers unchanged
rosbag2-compression-zstd	0.33.4 (build 30) → 0.33.4	compatible: no ABI change	headers unchanged
ur-controllers	6.1.0 (build 30) → 6.1.0	compatible: no ABI change	headers unchanged
ros-gz-sim-demos	3.0.10 (build 30) → 3.0.10	no libraries: nothing to compare
ros2-control	6.10.1 (build 30) → 6.10.1	no libraries: nothing to compare
rviz2	15.2.6 (build 30) → 15.2.6	no libraries: nothing to compare

Compared with the newest builds on https://repo.prefix.dev/robostack-lyrical/linux-64.

About this check

Each package this pull request built, compared with its newest released build (tools/abi_check.py, libabigail):

compatible / additions: the package can be bumped on its own;
soname / incompatible: the packages that depend on it need a rebuild (build number bump) in the same pull request.

Exported symbols are compared (the released packages are stripped, so struct layouts aren't); unchanged installed headers rule out layout and inline changes, changed ones are listed for review. Dependents to rebuild are the released ones linking to a changed library. A changed pin is part of the comparison.

Tobias-Fischer · 2026-10-24T05:55:47Z

Run B (547edaa Defender off, ca5d5f6 Spotlight off), per-recipe wall time against lyrical's build-branch jobs:

	recipes	before	after
linux-64 (control, unchanged)	7	334 s	308 s (−8%)
osx-arm64	4	291 s	254 s (−13%)
osx-64	4	471 s	339 s (−28%)
win-64	1	218 s	142 s (−35%)

On Windows, installs went from 95 s to 58 s and clean-up from 54 s to 21 s; compiling was unchanged. The Access is denied retries remain (MSVC's mspdbsrv.exe holding a handle, prefix-dev/rattler-build#2657), but they only take ~1 s. Most of the remaining time is file I/O on C:.

Next: run C (e46dea1 parallel builds, 362f46f estimate from timings), then run D (builds on D:) and run E (Dev Drive).

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Tobias-Fischer · 2026-10-24T06:19:16Z

Run C (e46dea1: rattler-build 0.77 with --parallel-builds 3:<cores>, 2 on Windows). Build step against run B, normalised by the recipes' measured times, since C rebuilt all 10 test packages:

	B	C	per unit of work
linux-64	309 s	284 s	−18%
linux-aarch64	256 s	185 s	−41%
osx-arm64	398 s	352 s	−25%
osx-64	571 s	654 s	−7%
win-64	843 s	1216 s	+9%

Windows has no jobserver, so two builds each run a full ninja on 4 cores and the slow C: drive. With 6–10 packages per platform these are rough. 362f46f's estimate ignores the parallelism so far (Linux: "6 min" for 3–5 min).

Next: run D (8db49c6, Windows builds on D:).

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Tobias-Fischer · 2026-10-24T06:38:47Z

Run D (8db49c6, Windows: builds, package cache and TEMP on D:). win-64 build step: 1216 s → 305 s (−75%), same 6 packages as run C. D: has 146 GB free (C: 82 GB).

Linux and macOS are unchanged from C, so this is a repeat (same 10 packages):

	C	D
linux-64	284 s	256 s	−10%
linux-aarch64	185 s	181 s	−2%
osx-arm64	352 s	344 s	−2%
osx-64	654 s	864 s	+32%

So Linux and osx-arm64 repeat to within ~10%, while osx-64's Intel runners vary by a third: its −28% (B) and −7% (C) are within that noise.

Next: run E (322d960, a Dev Drive instead of D:).

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Tobias-Fischer · 2026-10-24T07:10:09Z

Run E: no Dev Drives on windows-2022 (322d960: Format-Volume -DevDrive needs Windows 11 or Server 2025). A plain ReFS volume on D: (bfb3479) failed rattler-build's reindex with Access is denied writing repodata.json, so D: (8db49c6) it is.

Linux and macOS build step over the four identical runs C, D, E, E': linux-64 284/256/252/292 s, linux-aarch64 185/181/188/226 s, osx-arm64 352/344/360/333 s, osx-64 654/864 s.

Next: run F (af8d30d), the setup before this PR with the same test packages, as a same-day baseline.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Installing the build and host environments and deleting a package's work directory afterwards took 3-4x as long as compiling in lyrical's win-64 jobs (e.g. 377 s of installs and 186 s of clean-up against 127 s of builds, the clean-up ending in "Access is denied" retries): Defender scanning every file. The runners are thrown away after each job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Spotlight's indexer works through every file the environments and builds write; other projects saw it take one or two of a GitHub macOS runner's three or four cores (PennyLaneAI/catalyst: "xcode builds very slow if we don't turn off spotlight"). The runners are thrown away after each job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rattler-build 0.77's --parallel-builds builds outputs whose dependencies are ready side by side; on Linux and macOS they share a make/ninja jobserver with the runner's cores, so they don't oversubscribe it. Windows has no jobserver: two builds at a time. A PR's builds run in one rattler-build call per platform, so most of their per-recipe time (environment installs, configure, packaging) can overlap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ci_estimate.py counted every package to build at one flat per-platform average from jazzy's October rebuild (19 s on linux-64, 69 s on win-64), so rebuilding rtabmap was estimated like a message package. It now adds up each recipe's measured time (build_timings.timings(), the same times vinca packs the build jobs by); the flat averages stay for platforms without times and for the packages restored from the cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

GitHub's Windows runners have a faster second drive (D:, about 14k IOPS against C:'s 4.5k, chadgolden1/gh-runner-benchmarks); uv uses it for the same reason. The package cache moves with the builds so that rattler keeps hardlinking packages into the environments. Prints both drives' free space. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

build_unix.sh / build_win.bat call build-ci once with all of the job's recipes instead of once per recipe. build-ci then adds --parallel-builds (3 builds sharing a jobserver with the runner's cores on Linux and macOS, 2 on Windows) and --log-style json: parallel builds interleave their output, so it prints each line as "HH:MM:SS [recipe] message", and records each recipe's time from its first to its last line in its build span for packing the jobs. Locally (osx-arm64), the 20 test recipes of #154 build in 169 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A recipe's time measured while others build alongside is longer than its share of the job. Scaling all of them by wall time / their sum gives each recipe its share, which is what packing the jobs needs (and what the serial times were). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

They only repackage but, building alongside, mostly wait for the others: their measured times inflated the sum that the packages' times are scaled by. The timing files never keep them (ALIAS_SECONDS), so leave them out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Tobias-Fischer · 2026-10-24T07:41:02Z

Run F (af8d30d: the setup before this PR, same test packages). Build step against this PR's setup (mean of runs C, D, E, E′; Windows: run D):

	before	after
linux-64	337 s	271 s	−20%
linux-aarch64	263 s	195 s	−26%
osx-arm64	435 s	347 s	−20%
osx-64	1051 s	759 s	−28% (noisy)
win-64	1615 s	305 s	−81%

Next: run G+H (c953590): Windows one build at a time on D:, macOS with Spotlight on.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Two parallel builds were no faster on Windows (#154: 305 s against 282 s for one at a time, with the builds on D:), and there is no jobserver to keep them from oversubscribing the cores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Tobias-Fischer · 2026-10-24T08:12:23Z

Run G+H (c953590):

Windows, one build at a time on D: (G): 282 s, against 305 s with two in run D. Windows now builds one recipe at a time (aca5a18).
macOS, Spotlight on (H): osx-arm64 414 s, against 333–360 s with Spotlight off (four runs), so turning it off saves ~16%. osx-64 took 635 s against 654 and 864 s, which is within its noise.
Linux, unchanged: 238 s and 191 s, a fifth repeat.

osx-64's job failed only while saving the cache artifact (ENOTFOUND); its build had succeeded.

Next: run I, Defender on, on the final setup.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Tobias-Fischer · 2026-10-24T08:32:15Z

Run I (3dfe5c8, the final setup with Defender on):

win-64: 441 s, against 282 s with Defender off (run G), so turning it off still saves ~36% with the builds on D:.
osx-arm64: 485 s with the final setup (Spotlight off). This is the same image and the same 20 packages as the earlier runs at 333–360 s, so osx-arm64 varies more than those four runs showed. Run H's 414 s with Spotlight on is within that range: the Spotlight gain is unproven. It stays off as a cheap change others report helping.
Linux: 281 s and 196 s, within the earlier range.

Next: the final branch replaces the test commits.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rolling: snapshot of rosdistro master from 2026-10-16 (full rebuild), on the shared conda-forge pinning and the shared Qt6 mapping. - distro.yaml: robostack-rolling on prefix.dev and anaconda.org, the shared pinning; - build_number 32, the mutex without its own build number: 0.23.0 rolling_32; - package fixes for the new versions (patches, dependencies, pkg_additional_info), rviz 16 / tf2_ros 0.46, ur_robot_driver on ros2_control 6.12; - shared tests: std_msgs uses the environment's Python, longer smoke-test timeouts; - ci.yaml: evict_cache reset. [skip ci]: the build branches are regenerated by the merge of #154 right after, which changes shared files and so regenerates every distribution, this one included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Tobias-Fischer added the build:lyrical PR build: build only lyrical (with other build:<distro> labels: only those) label Oct 24, 2026

Tobias-Fischer force-pushed the windows-defender-off branch from 3a5a22c to 10b9347 Compare October 24, 2026 05:03

Tobias-Fischer changed the title ~~Windows: no Defender real-time scan in the build jobs~~ Windows: no Defender real-time scan; macOS: no Spotlight, in the build jobs Oct 24, 2026

Tobias-Fischer force-pushed the windows-defender-off branch from 10b9347 to ce59dd8 Compare October 24, 2026 05:19

Tobias-Fischer force-pushed the windows-defender-off branch from bfb3479 to af8d30d Compare October 24, 2026 07:09

Tobias-Fischer and others added 8 commits October 24, 2026 17:28

Tobias-Fischer changed the title ~~Windows: no Defender real-time scan; macOS: no Spotlight, in the build jobs~~ Faster CI: Windows builds on D:, Defender and Spotlight off, parallel builds Oct 24, 2026

Tobias-Fischer force-pushed the windows-defender-off branch from af8d30d to c953590 Compare October 24, 2026 07:40

Windows: one build at a time

aca5a18

Two parallel builds were no faster on Windows (#154: 305 s against 282 s for one at a time, with the builds on D:), and there is no jobserver to keep them from oversubscribing the cores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Tobias-Fischer force-pushed the windows-defender-off branch from c953590 to 3dfe5c8 Compare October 24, 2026 08:12

Tobias-Fischer force-pushed the windows-defender-off branch from 3dfe5c8 to aca5a18 Compare October 24, 2026 08:32

Windows PR builds: pixi's package cache on D: too

94cd474

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Tobias-Fischer force-pushed the windows-defender-off branch from c13d78d to 94cd474 Compare October 24, 2026 10:32

Tobias-Fischer marked this pull request as ready for review October 24, 2026 11:00

Tobias-Fischer merged commit 23f8412 into main Oct 24, 2026
12 checks passed

Tobias-Fischer deleted the windows-defender-off branch October 24, 2026 11:00

Tobias-Fischer mentioned this pull request Oct 24, 2026

rolling: septentrio_gnss_driver patch for 1.4.8-3; build-ci names unfinished builds #157

Open

Navigation Menu

Repository navigation

Faster CI: Windows builds on D:, Defender and Spotlight off, parallel builds - #154

Faster CI: Windows builds on D:, Defender and Spotlight off, parallel builds#154
Tobias-Fischer merged 10 commits into
mainfrom
windows-defender-off

Tobias-Fischer commented Oct 24, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 24, 2026 •

edited

Loading

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Tobias-Fischer commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Measurements

Before merging

Uh oh!

github-actions Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ABI check (linux-64)

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Tobias-Fischer commented Oct 24, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Tobias-Fischer commented Oct 24, 2026 •

edited

Loading

github-actions Bot commented Oct 24, 2026 •

edited

Loading