Skip to content

RISC-V: unroll the ZVL256B DGEMM kernel K-loop by two - #6168

Open
HiderX wants to merge 1 commit into
OpenMathLib:developfrom
HiderX:riscv-zvl256b-dgemm-unroll2
Open

HiderX wants to merge 1 commit into
OpenMathLib:developfrom
HiderX:riscv-zvl256b-dgemm-unroll2

Conversation

@HiderX

@HiderX HiderX commented Oct 25, 2026 •

Copy link
Copy Markdown

Summary

Unroll the K-loop of the ZVL256B DGEMM 8x8 micro-kernel by two, with the
two A vectors loaded before the FMA stream (A double-buffering). This is the
same transformation that #6054 applied to the ZVL256B SGEMM kernel.

Before this change, the main 8x8 pass advanced one k-step per iteration. Each
iteration did:

  • eight B scalar loads;
  • one 8-element A load (LMUL 2);
  • eight accumulating FMAs;
  • the B/A pointer bumps and the loop backedge.

Folding two k-steps into one iteration halves the k-counter updates, the
B/A pointer bumps and the backedges. It also issues both A loads up front.

As in the SGEMM kernel, the first k-step is already peeled ahead of the loop
(the vfmul calls that initialize the accumulators). The main pass is now

for (k = K; k > 2; k -= 2) { /* two k-steps */ }
for (; --k; )              { /* one k-step  */ }

This consumes exactly K steps for every K. Each accumulator still consumes the
k-steps in ascending order, so the per-column accumulation order is unchanged
and the results are bit-identical to the previous kernel. The M/N tail paths
are not touched.

In a default RISCV64_ZVL256B build, this source is compiled as dgemm_kernel,
dsymm_kernel and dtrmm_gemm_kernel, none of them with -DTRMMKERNEL.
DTRMM uses dtrmm_kernel_8x8_zvl256b.c, which is unchanged.

Performance

Setup:

  • SpacemiT K3, one X100 core (cpu 4, 2.2 GHz, fixed), single thread
    (OPENBLAS_NUM_THREADS=1), GCC 15.2.0.
  • Square cblas_dgemm, row-major, NoTrans/NoTrans, alpha = 1, beta = 0.
  • Each number is the median GFLOP/s of 40 timed calls per size: 4 separate
    processes per build, base and patched builds alternating, each process doing
    10 repetitions after 2 warm-up calls.
n develop (5bab605) patched change
256 13.39 14.05 +4.90%
512 12.10 12.56 +3.79%
1024 12.84 13.17 +2.58%
2048 12.81 13.30 +3.77%

At every size, the slowest patched process was faster than the fastest
unpatched one. As a control, SGEMM was measured the same way in the same job;
its kernel source is identical in both builds. It changed by -0.32% to +0.11%.

Testing

  • OpenBLAS test suite. Both trees were built on the board with
    make TARGET=RISCV64_ZVL256B NO_FORTRAN=1 NO_LAPACK=1 NO_LAPACKE=1 USE_OPENMP=0 NUM_THREADS=8; the default goal runs the tests. The test
    results are identical:

    • utest 141/141;
    • extension tests 1479/1479;
    • every BLAS/CBLAS test reports PASSED, with no failure.
  • Differential test against the unpatched library. 14400 cblas_dgemm
    calls, with padded leading dimensions, covering:

    • both storage orders;
    • all four transpose combinations;
    • M, N in {1, 3, 7, 8, 9, 16, 17, 33, 65, 129};
    • K in {1, 2, 3, 4, 5, 17, 128, 129, 300};
    • two (alpha, beta) pairs.

    A hash of every output buffer, padding included, is identical between the
    two builds.

  • Disassembly. The disassembled dgemm_kernel gains exactly the unrolled
    pass: +2 vle64.v, +16 fld, +16 vfmacc.vf.

This does not overlap with #5561 or with the draft #6066. The latter changes
only the tail paths of this file, and this patch applies cleanly on top of it.

Disclosure: an AI coding agent (Anthropic Claude) prepared this patch, its
validation runs and this description on behalf of the account owner. All
numbers above come from archived runs on the board.

Apply the same transformation as the ZVL256B SGEMM kernel (OpenMathLib#6054) to
the DGEMM 8x8 micro-kernel: fold two consecutive k-steps into one
iteration with both A vectors loaded before the FMA stream (A
double-buffering), and keep the one-step loop for the remainder. Each
accumulator still consumes the k-steps in ascending order, so results
are bit-identical to the previous kernel.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant