Skip to content

test: preserve image baselines across Matplotlib text renderers - #9201

Draft
agourakis82 wants to merge 2 commits into
Project-MONAI:devfrom
agourakis82:fix/matplotlib-311-image-baselines
Draft

agourakis82 wants to merge 2 commits into
Project-MONAI:devfrom
agourakis82:fix/matplotlib-311-image-baselines

Conversation

@agourakis82

@agourakis82 agourakis82 commented Oct 20, 2026 •

Copy link
Copy Markdown

Related to #9069.

Problem

With Matplotlib 3.11.2, the image comparisons in ThreadContainer.test_plot and three matshow3d cases fail against the existing reference PNGs. This patch selects references by renderer generation while keeping the existing comparison tolerances and legacy references.

Changes

  • Add get_matplotlib_baseline() to select the existing references before Matplotlib 3.11 and a separate matplotlib_3_11 directory from 3.11.0.dev0 onward.
  • Use that selector in the four affected image comparisons. Keep the historical test_samples RMS expectation only for the legacy reference; the modern reference must satisfy the normal image comparison.
  • Add tests for the version boundary, prereleases and the absence of fallback when a modern reference is missing.
  • Exclude the new Matplotlib-dependent selector module from the minimal-dependency suite, following the contribution guidelines.
  • Include four proposed modern reference PNGs. Production MONAI modules, dependency bounds and CI configuration are unchanged by this PR.

Validation and scope

On 2026-10-24, a fresh Ubuntu 24.04 / CPython 3.12.3 environment with Torch 2.8.0, torchvision 0.23.0 and Matplotlib 3.11.2 reproduced the four affected image failures. The native MONAI extension was built and imported, and dependency checks passed.

The command selected the zarr merger, FID, threadcontainer and matshow3d test modules:

python -m unittest -v tests.inferers.test_zarr_avg_merger tests.metrics.test_compute_fid_metric tests.data.test_threadcontainer tests.visualize.utils.test_matshow3d
  • Before: 32 tests, four image-comparison failures.
  • After applying the exact published renderer patch and adding tests.test_matplotlib_baseline to the command: 34 tests passed, with no skips.
  • All eight renderer files, including the four PNGs, were checked against the published Git blobs at 9cbaeadfda56acc8e8a6cf973a59cf28e9a6ffd7. The follow-up changes only the minimal-suite exclusion list.

This execution used a private composition based on #9140 at 4043f3bd332bf4867fe5a3ac80abb5257841a2c9, with a separate CI experiment, followed by the exact renderer patch. It is focused regression evidence, not a hosted-CI or complete-suite result for this standalone PR. The follow-up minimal-suite exclusion was checked for a unique entry and an existing module; the rendering tests were not rerun for that registration change. Hosted workflow approval remains a separate gate.

Remaining draft item: reference distribution

The current draft commits the four PNGs directly. CONTRIBUTING.md asks contributors to host binary test data outside the source repository, preferably in MONAI-extra-test-data releases, and reference it through tests/testing_data/data_config.json.

The references need to be moved to that distribution path before this draft is ready. Could these four files be hosted in a MONAI-extra-test-data release so the tests can use the project's download helpers?

Types of changes

  • Test-only change.
  • New selector tests.
  • Full integration suite passed for this standalone PR.
  • Hosted CI passed.
  • Reference distribution aligned with the project guidelines.

Technical summary prepared with OpenAI Codex assistance.

Matplotlib 3.11 changes text rendering, so references must track
renderer generations without weakening image comparisons.

Related to Project-MONAI#9069.

Assisted-by: OpenAI Codex
Signed-off-by: Demetrios Agourakis <demetrios@agourakis.med.br>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Register the optional-dependency test module alongside test_matshow3d in
the minimal-suite exclusion list, as required by CONTRIBUTING.md.

Verified one unique exclusion entry, an existing target module, and
git diff --check. This does not change the rendering code or references.

Assisted-by: OpenAI Codex
Signed-off-by: Demetrios Agourakis <demetrios@agourakis.med.br>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant