Skip to content

CI: add abicheck ABI scan alongside abi-compliance-checker - #542

Open
napetrov wants to merge 1 commit into
oneapi-src:masterfrom
napetrov:ci/abicheck
Open

napetrov wants to merge 1 commit into
oneapi-src:masterfrom
napetrov:ci/abicheck

Conversation

@napetrov

@napetrov napetrov commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds an advisory abicheck ABI scan next to the existing check-abi.yml (abi-compliance-checker). No ABI finding fails the PR, and check-abi.yml is unchanged. It reuses the build-quick.yml Release build instead of building the library again, and compares against ABI facts stored on the latest release instead of rebuilding the tag.

 build-quick.yml (pull_request)
   build-linux [ubuntu:22.04, ubuntu:24.04, ubi9]
     cmake Release, make, ctest
+    ubuntu:24.04 only: cmake --install → abi/{lib,include}   # loader + 2 layers, no NULL driver
+    upload-artifact abicheck-inst
+  abicheck (needs: build-linux)
+    latest semver tag → gh release download level-zero.abicheck-bundle.json.zst
+    abicheck compare <stored facts> new/      # one directory compare, python 3.15t
+    --policy append_only_dispatch_tables, --suppress .github/abicheck-suppressions.yml
+    → job summary: Breaking / Added / Suppressed per library + full markdown report
+    → ::warning / ::notice annotations, JSON + markdown artifact

+abicheck-baseline.yml (release published, or workflow_dispatch tag=…)
+  build the tag (same Release flags, ubuntu-24.04) → record facts → zstd (≈1.4 MB)
+  gh release upload, write-once (refuses to replace an existing asset)
+.github/abicheck.yml               # raises the decode limit for the stored facts (≈5M JSON nodes)
+.github/abicheck-suppressions.yml  # ZE_API_VERSION_CURRENT, same as #545's update_ze_api_version.py

Baseline: the PR compares against what shipped in the latest release, stored once per release as one asset. Until a release has that asset, the abicheck job only writes a note to the summary. To seed the current release: gh workflow run abicheck-baseline.yml -f tag=v1.34.0.

Relation to #545

#545 (merged) fixed the main ABICC false positives: it masks the ZE_API_VERSION_CURRENT bump, downgrades the register/offset rules, pins ACC/abi-dumper, caches the base dump and adds the abi-failure-approved override. With it, ABICC passes the #516 case. This PR does not overlap with that work. It now mirrors the CURRENT masking (a suppression file) and adds what ABICC still does not see: the two layers, offsets inside the _ze_dditable_t / _zes_dditable_t aggregates, zelTracerResetAllCallbacks, and the leaked loader C++ exports.

Rescan 2026-10-23 (level-zero master 746a50a, abicheck main ad964ac, Release, py3.15t)

Pair / config Breaking per library
v1.33.1 → v1.34.0, default 47 (+37 public API added, 0 removed)
+ append_only_dispatch_tables policy 43 (4 table-size growths accepted)
+ CURRENT suppression (this PR) 42
v1.34.0 → master 0 (NO_CHANGE)

The 42 that remain are 21 + 20 field-offset findings in the aggregates _ze_dditable_t / _zes_dditable_t, which embed the sub-tables by value, plus 1 size finding in _zes_dditable_driver_t (1600→1728) that the policy pattern misses. abicheck/abicheck#1759 asks for the policy to cover both. Until then the scan stays advisory. All earlier abicheck issues from this PR (#1706, #1708–#1710, #1713–#1715) are fixed upstream.

Comparison: abicheck vs. the current checker on v1.33.1 → v1.34.0 (before #545)

I ran both tools on the same two versions. For abi-compliance-checker I used this repo's check-abi.yml steps in an ubuntu:24.04 container.

Case abicheck abi-compliance-checker (check-abi.yml)
Libraries checked loader, validation layer, tracing layer libze_loader only
ZE_API_VERSION_CURRENT 1.18 → 1.19 breaking Medium, the only "problem" (95% binary compatibility)
_ze_command_queue_dditable_t 80→96 B, _zes_driver_dditable_t 40→72 B breaking (size change) Low warnings ("field added, size changed")
_ze_dditable_t / _zes_dditable_t aggregates: 41 field offsets moved, 3 sizes breaking not reported (not reachable from an exported public-header function)
zelTracerResetAllCallbacks declared in layers/zel_tracing_register_cb.h and in the tracing README, but implemented and exported nowhere (since at least v1.33.1) flagged once for the whole product not reported
Loader exports with no public declaration (singleton_factory_t, object_t, error_state, hashtable instantiations) flagged (exported_not_public; more with a -g build) not reported
Added public API 81 (loader) / 66 / 66 15 symbols listed
Removed public API 0 0
Verdict BREAKING, 47 gating findings per library 95%, exits non-zero

How to read this:

  • Both tools report the 1.19 dispatch-table growth and the *_CURRENT bump. These are true layout facts, but by Level Zero's design they are not breaks: the tables only grow at the end, and every zeGet*ProcAddrTable call passes a version. That is why Update L0 Loader w/L0 Spec 1.19.12 #516 failed check-abi.yml and was merged anyway. Both checks stay advisory until a policy encodes that rule (requested upstream: Built-in policy for append-only tables (Level Zero / Vulkan dispatch tables) abicheck/abicheck#1709). This PR now uses the upstream policy for the table growth and suppresses only CURRENT.
  • Only abicheck finds the zelTracerResetAllCallbacks gap, the loader's leaked C++ internals, and anything in the two layers.
  • The Release build (no -g) gives the same real findings as a -g debug build. It loses only 2 duplicate field-type findings and some exported_not_public detail, so the existing CI build is enough.

Evidence

Fork runs of this exact change. Test-only edits: repository_owner guards off, a push trigger, and the baseline built from v1.33.1 but attached to the v1.34.0 release, so the scan has a real diff. The test release and branch are deleted.

  • Baseline (run 39726793297): green in 4 min 11 s. It attached level-zero.abicheck-bundle.json.zst, 1.35 MB (133 MB uncompressed).

  • Write-once (run 39727218419): a second baseline run fails with level-zero.abicheck-bundle.json.zst already exists on v1.34.0 … delete it deliberately … to re-baseline, as intended.

  • PR scan, 10-23 head (run 40000543542): all build-quick.yml jobs green, abicheck job 3 min 17 s. The policy matched 4 findings and the suppression 1 per library:

    Library Verdict Breaking Added Suppressed
    libze_loader BREAKING 42 49 1
    libze_tracing_layer BREAKING 42 49 1
    libze_validation_layer BREAKING 42 49 1

    This table comes from a short jq step over the JSON, because the stored-facts markdown has no per-library counts. The full report still lists zelTracerResetAllCallbacks. The earlier run of the first version (39729517766) showed 47.

  • Interpreter: actions/setup-python installs CPython 3.15.1t (free-threaded).

  • actionlint 1.7.7 passes on both workflow files (10-23 head).

  • An earlier version of this PR built head, master and the tag separately (3 builds, 6 compare jobs). Results matched; this version drops the extra builds and the master comparison.

Merge Danger

Door: two-way

Four files: a new job in build-quick.yml, a new abicheck-baseline.yml, a 5-line .github/abicheck.yml and a 6-line .github/abicheck-suppressions.yml. check-abi.yml is untouched. Reverting removes the scan. The one lasting effect is the release asset level-zero.abicheck-bundle.json.zst. It is write-once, and a maintainer can delete it by hand.

Blast Radius: CI-only

  • build-quick.yml gains one job (about 3 min, after the Linux builds) and one install + upload step on the ubuntu:24.04 leg. The job needs: build-linux, so if any Linux leg fails, the scan is skipped.
  • The stored facts and the PR build must come from the same toolchain. Both use Ubuntu 24.04 GCC; if they ever differ, abicheck reports not_comparable and the job stays green. To fix that, re-record the baseline.
  • abicheck-baseline.yml holds contents: write, scoped to its one job. It appends an unsigned asset to the release. If all release assets must be signed (.sig), that step needs signing or should stay manual.
  • abicheck is pinned to a main commit, not a release tag, to get the latest scanner; keep both workflows on the same pin. The Action installs its own CastXML from conda-forge.
  • No PR comment, to keep CI read-only on PRs. The report goes to the job summary, annotations and an artifact.

🤖 Generated with Claude Code

@rwmcguir

Copy link
Copy Markdown
Contributor

See my PR #545 as it address a lot of these issues. I had these fixes present as part of the Spec repo, but I hadn't ported them over yet. Caching, filtering etc.. was already part of the items. Though a few of these may still be interesting.

build-quick.yml stages the ubuntu:24.04 Release install (loader and both
layers) and a new advisory abicheck job compares it, as one directory,
against ABI facts stored on the latest release. abicheck-baseline.yml
records those facts (one zstd JSON asset, write-once) when a release is
published or on dispatch. The job summary lists breaks, additions and
removals per library. check-abi.yml is unchanged.

Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
@github-actions

Copy link
Copy Markdown

Level Zero loader ABI check

last-release (v1.34.0): ✅ no ABI problems. Report

Creating compatibility report ...
Binary compatibility: 100%
Source compatibility: 100%
Total binary compatibility problems: 0, warnings: 0
Total source compatibility problems: 0, warnings: 0

master (master): ✅ no ABI problems. Report

Creating compatibility report ...
Binary compatibility: 100%
Source compatibility: 100%
Total binary compatibility problems: 0, warnings: 0
Total source compatibility problems: 0, warnings: 0

Workflow run

If an ABI change is intended, a reviewer with write access can accept it: leave an approval review on the PR head commit with the keyword abi-failure-approved in the review body. That re-runs the check.

@napetrov

Copy link
Copy Markdown
Contributor Author

Thanks, #545 covers most of what I raised for check-abi.yml. With it, ABICC passes the #516 case (100%, 0 problems), and caching, the CURRENT masking and the override are all done there. I've rebased this PR on top of it, and the two don't overlap. This PR now mirrors your CURRENT masking with a one-entry suppression file (.github/abicheck-suppressions.yml), so both tools treat that bump the same way.

Here is what abicheck still adds next to ABICC, measured on v1.33.1 → v1.34.0:

ABICC (check-abi.yml, after #545) abicheck (this PR)
Libraries checked libze_loader loader + validation layer + tracing layer
_ze_dditable_t / _zes_dditable_t aggregates (embed sub-tables by value; 41 field offsets moved in 1.19) not reported (not reachable from an exported function) reported
zelTracerResetAllCallbacks: declared in layers/zel_tracing_register_cb.h and the README, but exported by no library (since at least v1.33.1) not reported flagged as "declared, exported nowhere": callers get a link error
Loader exports with no public declaration (singleton_factory_t, object_t, hashtable instantiations) not reported flagged as exported_not_public
Added / removed public API 15 symbols listed full lists per library (37 added, 0 removed)
Dispatch-table growth Low warnings ("field added, size changed") for _ze_command_queue_dditable_t / _zes_driver_dditable_t a named policy (append_only_dispatch_tables) accepts tail growth of *_dditable_t only; any other struct growing still breaks
Baseline ABI dump of the base built in CI, then cached facts recorded once per release (1.35 MB asset), so no rebuild of the old tag
Cost separate build + abi-dumper reuses the build-quick.yml Release build; about 3 min, advisory

Two of these are real defects, not noise: the missing zelTracerResetAllCallbacks and the leaked C++ internals in the loader's export table. Neither shows up in ABICC.

Still open on the abicheck side: the policy doesn't yet cover the offsets inside the aggregates or _zes_dditable_driver_t (abicheck/abicheck#1759). That's why the job is advisory and never fails a PR. With your rules plus this policy, both tools would agree on what "by design" means for L0.

Glad to trim it further if you'd like only part of this, e.g. just the layer coverage and the export checks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants