Repository navigation
Conversation
Apply the same transformation as the ZVL256B SGEMM kernel (OpenMathLib#6054) to the DGEMM 8x8 micro-kernel: fold two consecutive k-steps into one iteration with both A vectors loaded before the FMA stream (A double-buffering), and keep the one-step loop for the remainder. Each accumulator still consumes the k-steps in ascending order, so results are bit-identical to the previous kernel.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unroll the K-loop of the ZVL256B DGEMM 8x8 micro-kernel by two, with the
two A vectors loaded before the FMA stream (A double-buffering). This is the
same transformation that #6054 applied to the ZVL256B SGEMM kernel.
Before this change, the main 8x8 pass advanced one k-step per iteration. Each
iteration did:
Folding two k-steps into one iteration halves the k-counter updates, the
B/A pointer bumps and the backedges. It also issues both A loads up front.
As in the SGEMM kernel, the first k-step is already peeled ahead of the loop
(the
vfmulcalls that initialize the accumulators). The main pass is nowThis consumes exactly K steps for every K. Each accumulator still consumes the
k-steps in ascending order, so the per-column accumulation order is unchanged
and the results are bit-identical to the previous kernel. The M/N tail paths
are not touched.
In a default RISCV64_ZVL256B build, this source is compiled as
dgemm_kernel,dsymm_kernelanddtrmm_gemm_kernel, none of them with-DTRMMKERNEL.DTRMM uses
dtrmm_kernel_8x8_zvl256b.c, which is unchanged.Performance
Setup:
(
OPENBLAS_NUM_THREADS=1), GCC 15.2.0.cblas_dgemm, row-major, NoTrans/NoTrans, alpha = 1, beta = 0.processes per build, base and patched builds alternating, each process doing
10 repetitions after 2 warm-up calls.
At every size, the slowest patched process was faster than the fastest
unpatched one. As a control, SGEMM was measured the same way in the same job;
its kernel source is identical in both builds. It changed by -0.32% to +0.11%.
Testing
OpenBLAS test suite. Both trees were built on the board with
make TARGET=RISCV64_ZVL256B NO_FORTRAN=1 NO_LAPACK=1 NO_LAPACKE=1 USE_OPENMP=0 NUM_THREADS=8; the default goal runs the tests. The testresults are identical:
Differential test against the unpatched library. 14400
cblas_dgemmcalls, with padded leading dimensions, covering:
A hash of every output buffer, padding included, is identical between the
two builds.
Disassembly. The disassembled
dgemm_kernelgains exactly the unrolledpass: +2
vle64.v, +16fld, +16vfmacc.vf.This does not overlap with #5561 or with the draft #6066. The latter changes
only the tail paths of this file, and this patch applies cleanly on top of it.
Disclosure: an AI coding agent (Anthropic Claude) prepared this patch, its
validation runs and this description on behalf of the account owner. All
numbers above come from archived runs on the board.