Repository navigation
runtime: reject kselftest passes without reported cases - #3247
Merged
nuclearcat merged 1 commit intoOct 21, 2026
Merged
Conversation
A completed LAVA job can contain only successful setup results after test signals are lost. A pull-lab summary can likewise claim success without supplying any cases. Both callbacks currently retain a passing job result. Mark these kselftest jobs incomplete rather than claiming test coverage. Preserve boot-only baseline behavior and genuine reported case results. Use the existing Test error code: it is accepted by the API and maps to ERROR in the KCIDB bridge without a coordinated schema deployment. Add regressions for absent and empty LAVA suites, setup-only baseline jobs, missing pull-lab cases, real failures behind passing summaries, and API error-code validation. Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
Member
Author
|
LAVA job 23252115: 191 kselftest cases passed, plus two setup checks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A completed LAVA job can contain only successful setup results after its test signals are lost. Pull-lab summaries can also report success without supplying cases. Mark these kselftest jobs incomplete when no cases are reported, preserving genuine case results and setup-only baseline jobs. Use the existing
Testerror code so the API accepts the result and the KCIDB bridge reportsERROR.Observed in production: a coredump ramdisk job was marked pass with only setup results, although its serial log reported a shard failure. This guard prevents an empty result from claiming coverage; it does not reconstruct test signals lost by the lab.
Validation: all 194 unit tests pass, including missing/empty LAVA suites, missing pull-lab cases, reported failures behind passing summaries, baseline behavior, and API error-code validation. CI-pinned Ruff lint and format checks pass.
make teststops at an existing mypy error inkernelci/api/latest.py:181, also reproduced on untouched upstream/main.