kpatch-build, lib.sh: add kernel variant support by usebeforefree · Pull Request #1530 · dynup/kpatch

usebeforefree · 2026-10-16T08:36:52Z

At the moment kpatch-build doesn't support kernel variants (e.g., +64k , +rt_64k), this PR aims to add support for kpatch-build and testing for them. Tested on kernel-64k-aarch64.

Cc: @joe-lawrence

joe-lawrence · 2026-10-19T19:01:55Z

@usebeforefree - thanks for the PR. The parsing and code looks mostly good, I have a few cosmetic suggestions:

Naming: I'd prefer the term used in the kernel specfile: VARIANT / variant (not FLAVOR / flavor)
Tabs vs. spaces: use tab convention in lib.sh
Commit message:
- add prefix to subject line: "kpatch-build, lib.sh: "
- note that these variants are for RHEL-derived distributions
- since I assume this is work related, we usually Signed-off-by with our work email

Also, at some point would you later update the testing farm / packit CI to run aarch64-64k ?

usebeforefree · 2026-10-20T10:11:20Z

Thanks for the review @joe-lawrence !

Applied all the suggestions.

(not FLAVOR / flavor)

I used the flavor term because I couldn't find the proper term, and saw this one used in kpatch-build:

kpatch/kpatch-build/kpatch-build

Lines 981 to 988 in 74ac9ff

    
           # handle flavor extension on Photon ex) -rt, -esx 
        
           if [[ "$DISTRO" = photon ]]; then 
        
           	KREL="${ARCHVERSION#*-}" 
        
           	# strip rt patchset version if present 
        
           	# remove trailing -flavor and starting -rt### patchset 
        
           	KREL="${KREL%-*}" 
        
           	KREL="${KREL#*-}"

I can find-and-replace them if we'd like to use the proper term.

Also, at some point would you later update the testing farm / packit CI to run aarch64-64k ?

I will do this in a separate PR, otherwise we'd create circular dependency within them.

joe-lawrence · 2026-10-20T13:29:18Z

(not FLAVOR / flavor)

I used the flavor term because I couldn't find the proper term, and saw this one used in kpatch-build:

kpatch/kpatch-build/kpatch-build

Lines 981 to 988 in 74ac9ff

# handle flavor extension on Photon ex) -rt, -esx

if [[ "$DISTRO" = photon ]]; then

KREL="${ARCHVERSION#*-}"

# strip rt patchset version if present

# remove trailing -flavor and starting -rt### patchset

KREL="${KREL%-*}"

KREL="${KREL#*-}"

I can find-and-replace them if we'd like to use the proper term.

You can leave the Photon "flavor" references in place. I have no idea if that's what that distribution uses that naming convention, I just thought for RHEL-based releases we should adopt what its kernel specfile is already using. (Naming things is already hard enough :)

Also, at some point would you later update the testing farm / packit CI to run aarch64-64k ?

I will do this in a separate PR, otherwise we'd create circular dependency within them.

Sounds good.

joe-lawrence · 2026-10-20T13:30:32Z

@Tomcat-42 @ykopkova : I have no idea how to add @usebeforefree, or anybody for that matter, to the testing farm access. The current results say, "usebeforefree can't run tests internally". How can we add him? Thanks.

usebeforefree · 2026-10-20T14:57:08Z

Update: I would actually like to enable the packit tests in this PR, working on it in the meantime.

ykopkova · 2026-10-20T15:17:57Z

@Tomcat-42 @ykopkova : I have no idea how to add @usebeforefree, or anybody for that matter, to the testing farm access. The current results say, "usebeforefree can't run tests internally". How can we add him? Thanks.

I "think" only project maintainers or members can trigger packit ci testing.

usebeforefree · 2026-10-21T12:13:36Z

Update:

I came up with this solution for adding the kernel-64k-aarch64 test case. I've added a commit that adds the case and targets a test to be added inside of kernel-tests that is defined here:

https://gitlab.com/redhat/centos-stream/tests/kernel/test-plans/-/merge_requests/233/diffs?commit_id=37b4f38573c391ea535934e5a66782253ef20122

I see the test case is called in this PR, someone just needs to figure out a way to make the pipeline accept it. Though we firstly need to target the test-plans branch above mentioned, so we can test it out before merging. What's a good way to do this?

The easiest certainly would be:

force pushing changes to make packit target the new kernel-tests branch:

diff --git a/.packit.yml b/.packit.yml
index 1939f6f..905a82c 100644
--- a/.packit.yml
+++ b/.packit.yml
@@ -61,8 +61,8 @@ _:
     skip_build: true
     manual_trigger: false
     tmt_plan: kpatch-integration-rhel/kernel-64k
-    fmf_url: https://gitlab.com/redhat/centos-stream/tests/kernel/test-plans.git
-    fmf_ref: main
+    fmf_url: https://gitlab.com/mkostic1/test-plans.git
+    fmf_ref: kernel-64k-aarch64-integration

run the packit pipeline, make sure it passes
merge the kernel-tests MR
force push back the packit target to main
merge this PR

Note to self: the same branch shenanigans could be done with test-plans as well (targeting a specific kernel-tests branch), so we don't have to merge anything before making sure things actually work.

Side question:

Should we update other tests to target rhel-10.3?

joe-lawrence · 2026-10-21T14:50:47Z

Update:

I came up with this solution for adding the kernel-64k-aarch64 test case. I've added a commit that adds the case and targets a test to be added inside of kernel-tests that is defined here:

https://gitlab.com/redhat/centos-stream/tests/kernel/test-plans/-/merge_requests/233/diffs?commit_id=37b4f38573c391ea535934e5a66782253ef20122

I see the test case is called in this PR, someone just needs to figure out a way to make the pipeline accept it. Though we firstly need to target the test-plans branch above mentioned, so we can test it out before merging. What's a good way to do this?

The easiest certainly would be:

force pushing changes to make packit target the new kernel-tests branch:
diff --git a/.packit.yml b/.packit.yml
index 1939f6f..905a82c 100644
--- a/.packit.yml
+++ b/.packit.yml
@@ -61,8 +61,8 @@ _:
     skip_build: true
     manual_trigger: false
     tmt_plan: kpatch-integration-rhel/kernel-64k
-    fmf_url: https://gitlab.com/redhat/centos-stream/tests/kernel/test-plans.git
-    fmf_ref: main
+    fmf_url: https://gitlab.com/mkostic1/test-plans.git
+    fmf_ref: kernel-64k-aarch64-integration
run the packit pipeline, make sure it passes

merge the kernel-tests MR

force push back the packit target to main

merge this PR

Note to self: the same branch shenanigans could be done with test-plans as well (targeting a specific kernel-tests branch), so we don't have to merge anything before making sure things actually work.

You know more than me about the testing :D, so I'd defer to Yulia / Pablo about this.

Side question:

Should we update other tests to target rhel-10.3?

The next sprint we should be updating/pushing out final integration tests for the next y-stream releases. I think that's a pre-req for what you're asking here?

ykopkova · 2026-10-21T14:52:56Z

/packit test

usebeforefree · 2026-10-21T15:31:53Z

/packit test

Tomcat-42 · 2026-10-21T15:34:10Z

/packit test

usebeforefree · 2026-10-21T15:54:22Z

not ready for merging, the 64k test is pointed to a custom branch

usebeforefree · 2026-10-21T15:54:36Z

/packit test

usebeforefree · 2026-10-22T07:46:59Z

/packit test

usebeforefree · 2026-10-22T12:20:04Z

The tests for all arches except kernel-64k-aarch64 are passing, so no regressions. The failing test I introduced in this PR is due to the master branch of kpatch not including integration-rhel10.3 tests. Which I will add in a new PR.

The only thing left really here is an issue I discovered along the way. The TLDR is that with this proposed fix you can't use the kpatch-build -r aka --sourcerpm argument with a kernel variant. Not only you cannot use it, it may introduce issues as it can produce bad results, because it will default to the non variant kernel. That's because the --sourcerpm does not describe the kernel variant, so we can't derive it.

kpatch/kpatch-build/kpatch-build

Lines 916 to 930 in 74ac9ff

    
           if [[ -n "$SRCRPM" ]]; then 
        
           	if  [[ -n "$ARCHVERSION" ]]; then 
        
           		warn "--archversion is incompatible with --sourcerpm" 
        
           		exit 1 
        
           	fi 
        
           	rpmname="$(basename "$SRCRPM")" 
        
           	if [[ "$DISTRO" = photon ]]; then 
        
           		ARCHVERSION="${rpmname%.src.rpm}" 
        
           	else 
        
           		ARCHVERSION="${rpmname%.src.rpm}.$(uname -m)" 
        
           	fi 
        
           	ARCHVERSION="${ARCHVERSION#kernel-}" 
        
           	ARCHVERSION="${ARCHVERSION#alt-}" 
        
           	ARCHVERSION="${ARCHVERSION#linux-}" 
        
           fi

usebeforefree · 2026-10-22T13:16:02Z

My proposal for how to fix the issue when calling -r is either:

Detecting kernel variant and exiting

	if  [[  "$(uname -r)" =~ \+(.*)$ ]]; then
		warn "kernel variants are incompatible with --sourcerpm"
		exit 1
	fi

Detecting kernel variant and appending it to the $SRCRPM derived ARCHVERSION:

   if [[ "$(uname -r)" =~ \+(.*)$ ]]; then
   	ARCHVERSION="${ARCHVERSION}+${BASH_REMATCH[1]}"
   fi

No idea what would be the better option. Exiting seems safer in any case, but I don't have the experience of how using kpatch-build to say if 2. solution would be viable.

joe-lawrence · 2026-10-23T14:29:23Z

My proposal for how to fix the issue when calling -r is either:

Detecting kernel variant and exiting
	if  [[  "$(uname -r)" =~ \+(.*)$ ]]; then
		warn "kernel variants are incompatible with --sourcerpm"
		exit 1
	fi
Detecting kernel variant and appending it to the $SRCRPM derived ARCHVERSION:
   if [[ "$(uname -r)" =~ \+(.*)$ ]]; then
   	ARCHVERSION="${ARCHVERSION}+${BASH_REMATCH[1]}"
   fi
No idea what would be the better option. Exiting seems safer in any case, but I don't have the experience of how using kpatch-build to say if 2. solution would be viable.

Here is how the kpatch-patch.spec runs kpatch-build:

# ... close your eyes, kids, it's .specfile voodoo ...
kpatch-build/kpatch-build -n %{patchmodname} -r %{SOURCE10} -v $vmlinux --skip-cleanup $patch_sources

I see, that's an interesting problem: the user initiates a kpatch-build and specifies the source rpm. Up until now, we could assume that the user was interested in the same-arch (non-variant) configuration from the source rpm. With this arm64 64k variant, that assumption may not always hold true. (BTW, I think this may be a similar problem if one wanted to build for modern RHEL realtime variant as well.)

I think the "right" answer is probably an additional --variant option that complements --sourcerpm. That way if one wants to build kpatches for both variants from a kernel-1234.aarch64 boot, they can run one w/o and one w/--variant 64k or something like that. (Or similarly for -rt if that ever needed to be supported.)

Does that capture the last hurdle and work as a solution?

usebeforefree · 2026-10-23T15:07:30Z

I think the proposal would work, but only if running kpatch-build on ex. kernel-64k without specifying --variant 64k doesn't cause potential major failures.

I don't have the expertise to know this, what I do have are logs of tests that failed due to this exact situation, and an LLM analysis:

A mismatch can at least cause a user to waste time on building the wrong kernel (see :: [ 14:09:02 ] :: timestamp in the logs), and also poison the cache with the wrong kernel config. Worst case scenario a buggy module that may cause a kernel panic?

If the above is true, maybe we can just add an internal check for mismatch between running system variant and (non)passed variant?

edit: I am confused about the usage of kpatch-build across different host and target systems, so feel free to just say if all this above is not really relevant, and I'll proceed to implement --variant.

joe-lawrence · 2026-10-23T17:59:55Z

If the above is true, maybe we can just add an internal check for mismatch between running system variant and (non)passed variant?

The user can build with a cross compiler, or in the case of arm64, either the ordinary or variant regardless of which one they are currently booted/installed.

usebeforefree · 2026-10-27T17:11:28Z

Rebased to include rhel10.3 integration tests, added the --variant argument. Fixed the failing tests in kernel-tests and called it here:

https://artifacts.osci.redhat.com/testing-farm/679ddddf-e4ee-4872-b703-6c4441848d8f

Expecting the tests to pass and the variant argument to get called from this line:

https://gitlab.com/mkostic1/kernel-tests/-/blob/aarch64-64k-page-support/general/kpatch/include/lib_build.sh#L133

Will request a review after I confirm both the integration, and tier2 tests pass.

joe-lawrence · 2026-10-28T14:30:13Z

Thanks, the -V/--variant + ARCHVERSION + suffix under -r is the right fix for the shared-SRPM case.

Two follow-ups before this is ready:

--variant requires --sourcerpm (not vice versa)

--sourcerpm alone should keep working. --variant without --sourcerpm should error out (currently it can pick a variant config while leaving ARCHVERSION/vmlinux/EXTRAVERSION on the non-variant release).

Untested (after option parsing / distro setup, once both are known):

if [[ -n "$VARIANT" && -z "$SRCRPM" ]]; then
    die "--variant requires --sourcerpm"
fi

This is the same pattern as the existing --oot-module requires --oot-module-src check. No need to spell the pairing out in the man page as it doesn’t document other option couplings either (usage can stay a one-liner like the other flags.)

Clearer error for a bogus --variant

There’s no check, so a bad value usually dies later as "kernel-debuginfo-…+bogus not installed or can't find config file." After the SRPM is unpacked and you’re selecting the config under configs/, we could fail explicitly, e.g.:

elif [[ -n "$VARIANT" ]]; then
    cfg="$KERNEL_SRCDIR/configs/kernel$ALT-$KVER-$ARCH-$VARIANT.config"
    [[ -e "$cfg" ]] || die "kernel variant '$VARIANT' not found (expected $(basename "$cfg"))"
    [[ -z "$CONFIGFILE" ]] && CONFIGFILE="$cfg"

Note: the debuginfo/vmlinux check still runs before unpack, so without -v a bad variant may still hit that path first. The config check above covers the post-unpack case. Special-casing the earlier debuginfo die when --variant was set is optional.

If this is starting to feel like scope creep, feel free to remove the variant / sourcerpm option for now and we can tackle in a subsequent MR to unblock testing (if that's possible). Thanks for working on this!

Add kpatch-build kernel variant argument (e.g., 64k , rt-64k), and integration tests. This effort adds support for RHEL-derived distributions. Signed-off-by: Marko Kostić <mkostic@redhat.com>

Signed-off-by: Marko Kostić <mkostic@redhat.com>

usebeforefree · 2026-10-29T11:19:26Z

Applied all suggested changes.

--variant can be only called with --sourcerpm.

Other paths set ARCHVERSION which contains the kernel variant, if it exists, which is later used to set VARIANT.

tests:

https://artifacts.osci.redhat.com/testing-farm/5c49f894-b0c8-4296-873b-4ded30394de6/

usebeforefree changed the title ~~add kernel flavor support~~ add kernel variant support Oct 20, 2026

usebeforefree force-pushed the kernel-flavor-support branch 2 times, most recently from 12cc263 to 3e73301 Compare October 20, 2026 08:59

usebeforefree changed the title ~~add kernel variant support~~ kpatch-build, lib.sh: add kernel variant support Oct 20, 2026

usebeforefree force-pushed the kernel-flavor-support branch from 3e73301 to 1ce2d1a Compare October 20, 2026 09:25

usebeforefree force-pushed the kernel-flavor-support branch from f37de49 to 38fb136 Compare October 21, 2026 15:30

usebeforefree force-pushed the kernel-flavor-support branch from 38fb136 to bc8ebe1 Compare October 21, 2026 15:53

usebeforefree marked this pull request as draft October 21, 2026 15:56

usebeforefree force-pushed the kernel-flavor-support branch from bc8ebe1 to 1a0a39d Compare October 22, 2026 07:44

usebeforefree force-pushed the kernel-flavor-support branch from 1a0a39d to 29cf1af Compare October 22, 2026 07:49

usebeforefree mentioned this pull request Oct 23, 2026

test/integration, .packit.yml: add rhel10.3 tests #1531

Merged

usebeforefree force-pushed the kernel-flavor-support branch 3 times, most recently from 4272f3c to 5bd14a1 Compare October 27, 2026 16:45

usebeforefree force-pushed the kernel-flavor-support branch from 5bd14a1 to 1b7888c Compare October 27, 2026 17:27

usebeforefree added 2 commits October 29, 2026 11:19

kpatch-build, lib.sh, man: add kernel variant support

bfad263

Add kpatch-build kernel variant argument (e.g., 64k , rt-64k), and integration tests. This effort adds support for RHEL-derived distributions. Signed-off-by: Marko Kostić <mkostic@redhat.com>

.packit.yml: add kernel-64k-aarch64 integration test case

1c1a8a8

Signed-off-by: Marko Kostić <mkostic@redhat.com>

usebeforefree force-pushed the kernel-flavor-support branch from 1b7888c to 1c1a8a8 Compare October 29, 2026 10:22

usebeforefree requested a review from joe-lawrence October 29, 2026 14:38

Conversation

usebeforefree commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joe-lawrence commented Oct 19, 2026

Uh oh!

usebeforefree commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joe-lawrence commented Oct 20, 2026

Uh oh!

joe-lawrence commented Oct 20, 2026

Uh oh!

usebeforefree commented Oct 20, 2026

Uh oh!

ykopkova commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

usebeforefree commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joe-lawrence commented Oct 21, 2026

Uh oh!

ykopkova commented Oct 21, 2026

Uh oh!

usebeforefree commented Oct 21, 2026

Uh oh!

Tomcat-42 commented Oct 21, 2026

Uh oh!

usebeforefree commented Oct 21, 2026

Uh oh!

usebeforefree commented Oct 21, 2026

Uh oh!

usebeforefree commented Oct 22, 2026

Uh oh!

usebeforefree commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

usebeforefree commented Oct 22, 2026

Uh oh!

joe-lawrence commented Oct 23, 2026

Uh oh!

usebeforefree commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joe-lawrence commented Oct 23, 2026

Uh oh!

usebeforefree commented Oct 27, 2026

Uh oh!

joe-lawrence commented Oct 28, 2026

Uh oh!

usebeforefree commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

usebeforefree commented Oct 16, 2026 •

edited

Loading

usebeforefree commented Oct 20, 2026 •

edited

Loading

ykopkova commented Oct 20, 2026 •

edited

Loading

usebeforefree commented Oct 21, 2026 •

edited

Loading

usebeforefree commented Oct 22, 2026 •

edited

Loading

usebeforefree commented Oct 23, 2026 •

edited

Loading