Repository navigation
fix(xc): distribute Libxc batches across OpenMP threads - #8255
Conversation
AsTonyshment
left a comment
There was a problem hiding this comment.
Nice fix! schedule(static, nr_batch_size) is insane, how did code like this end up here? I suspect the original developer confused the number of grid points per batch with the number of OpenMP loop iterations per chunk. This is a pretty serious performance bug that's surprisingly easy to overlook in the pre-AI-agent era...
ABACUS-PR-BOT
left a comment
There was a problem hiding this comment.
No confirmed implementation defect found in the inspected paths. The inline comment covers threaded test coverage.
Static review of 90f5c253 against d20090cc: whitespace and governance checks passed. No build or runtime verification was performed; benchmark results remain author-reported.
PS: After tracking down, I found that #7130 is to blame for this :( |
What's changed?
Libxc loops already process 1024 grid points per iteration. Scheduling another 1024 iterations per OpenMP chunk can leave most threads idle. Use
schedule(static)in the four existing loops to distribute these batches across threads.The change is limited to four OpenMP pragmas in the shared Libxc code used by LCAO and PW. Functional evaluation and numerical settings are unchanged.
Performance
LCAO HSE calculations on displaced Si32 and CsPbI3 (5 atoms), using one V100, one MPI rank, four physical CPU cores/OpenMP threads and serial BLAS on SAI. Both versions use Libxc 7.0.0, cuSOLVER diagonalization and CPU LibRI contractions. The baseline is unmodified
d20090cc137f.Ranges cover two paired runs in separate allocations on shared nodes, with reversed execution order. Ratios are calculated within each pair; SCF iteration counts match. Complete time includes process startup. The Libxc and
Potential::cal_vefftimers are nested and reported separately.Unit Tests and/or Case Tests for my changes
The full CUDA executable build and existing XC tests passed with one and four OpenMP threads:
The four benchmark comparisons and a one-thread Si32 control converged with matching iteration counts. Maximum differences were 8.4e-12 eV/atom and 6.0e-10 kbar; forces agreed at the recorded output precision. These comparisons establish agreement with the baseline.
Inputs, timings, numerical comparisons and test logs. Benchmark executables enable identical existing timer JSON output; this instrumentation is excluded from the change.
Linked Issue
Supersedes the Libxc scheduling part of #8193 and can be merged independently of the RI optimizations and #8169.
Governance Notes
No documentation update required: this only changes scheduling in shared XC evaluation, with no INPUT, interface or dependency changes.