Skip to content

fix(xc): distribute Libxc batches across OpenMP threads - #8255

Merged
mohanchen merged 1 commit into
deepmodeling:developfrom
Stardust0831:perf/libxc-omp-batches
Nov 2, 2026
Merged

mohanchen merged 1 commit into
deepmodeling:developfrom
Stardust0831:perf/libxc-omp-batches

Conversation

@Stardust0831

Copy link
Copy Markdown
Collaborator

What's changed?

Libxc loops already process 1024 grid points per iteration. Scheduling another 1024 iterations per OpenMP chunk can leave most threads idle. Use schedule(static) in the four existing loops to distribute these batches across threads.

The change is limited to four OpenMP pragmas in the shared Libxc code used by LCAO and PW. Functional evaluation and numerical settings are unchanged.

Performance

System Stage Baseline (s) This PR (s) Paired speedup
Si32 Libxc evaluation 77.87–88.66 19.51–22.70 3.91–3.99x
Si32 Effective potential 98.46–110.97 40.89–46.17 2.40–2.41x
Si32 Complete SCF + forces + stress 182.22–203.71 122.53–138.02 1.48–1.49x
CsPbI3 Libxc evaluation 42.56–42.75 10.29–10.86 3.94–4.13x
CsPbI3 Effective potential 47.81–48.09 15.34–16.11 2.98–3.12x
CsPbI3 Complete SCF + forces + stress 424.48–426.14 391.86–396.49 1.07–1.08x

LCAO HSE calculations on displaced Si32 and CsPbI3 (5 atoms), using one V100, one MPI rank, four physical CPU cores/OpenMP threads and serial BLAS on SAI. Both versions use Libxc 7.0.0, cuSOLVER diagonalization and CPU LibRI contractions. The baseline is unmodified d20090cc137f.

Ranges cover two paired runs in separate allocations on shared nodes, with reversed execution order. Ratios are calculated within each pair; SCF iteration counts match. Complete time includes process startup. The Libxc and Potential::cal_veff timers are nested and reported separately.

Unit Tests and/or Case Tests for my changes

The full CUDA executable build and existing XC tests passed with one and four OpenMP threads:

cmake --build build --target abacus_std_gpu MODULE_HAMILT_XCTest_VXC --parallel 24
OMP_NUM_THREADS=1 ctest --test-dir build -V -R '^(MODULE_HAMILT_XCTest_VXC)$'
OMP_NUM_THREADS=4 ctest --test-dir build -V -R '^(MODULE_HAMILT_XCTest_VXC)$'

The four benchmark comparisons and a one-thread Si32 control converged with matching iteration counts. Maximum differences were 8.4e-12 eV/atom and 6.0e-10 kbar; forces agreed at the recorded output precision. These comparisons establish agreement with the baseline.

Inputs, timings, numerical comparisons and test logs. Benchmark executables enable identical existing timer JSON output; this instrumentation is excluded from the change.

Linked Issue

Supersedes the Libxc scheduling part of #8193 and can be merged independently of the RI optimizations and #8169.

Governance Notes

No documentation update required: this only changes scheduling in shared XC evaluation, with no INPUT, interface or dependency changes.

@AsTonyshment AsTonyshment left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix! schedule(static, nr_batch_size) is insane, how did code like this end up here? I suspect the original developer confused the number of grid points per batch with the number of OpenMP loop iterations per chunk. This is a pretty serious performance bug that's surprisingly easy to overlook in the pre-AI-agent era...

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No confirmed implementation defect found in the inspected paths. The inline comment covers threaded test coverage.

Static review of 90f5c253 against d20090cc: whitespace and governance checks passed. No build or runtime verification was performed; benchmark results remain author-reported.

@AsTonyshment

Copy link
Copy Markdown
Collaborator

Nice fix! schedule(static, nr_batch_size) is insane, how did code like this end up here? I suspect the original developer confused the number of grid points per batch with the number of OpenMP loop iterations per chunk. This is a pretty serious performance bug that's surprisingly easy to overlook in the pre-AI-agent era...

PS: After tracking down, I found that #7130 is to blame for this :(

@mohanchen mohanchen added GPU & DCU & HPC GPU and DCU and HPC related any issues Refactor Refactor ABACUS codes Performance Issues related to fail running ABACUS labels Nov 2, 2026
@mohanchen
mohanchen merged commit 036b641 into deepmodeling:develop Nov 2, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

GPU & DCU & HPC GPU and DCU and HPC related any issues Performance Issues related to fail running ABACUS Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants