Skip to content

Feature: rework out_app_flag as a three-mode output controller with unified output directories, append semantics, and provenance headers - #8159

Open
mohanchen wants to merge 112 commits into
deepmodeling:developfrom
mohanchen:2026-10-16-b
Open

mohanchen wants to merge 112 commits into
deepmodeling:developfrom
mohanchen:2026-10-16-b

Conversation

@mohanchen

@mohanchen mohanchen commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR reworks out_app_flag from a boolean into a three-mode integer switch
and, on top of that, reorganizes ionic-step outputs into unified directories,
fixes several MPI/MD output bugs, and adds machine-readable provenance headers
to user-facing output files.

out_app_flag semantics (bool -> int)

  • out_app_flag is now an int with three modes:
    • 0 (default): legacy per-step file names and layout (unchanged behavior).
    • 1: fixed file names; per-ionic-step records are appended to a single
      accumulated file (cube frames, PW/LCAO wavefunctions, band output, DOS/PDOS,
      DM(k), T(R)/R(R)/dH/dS sparse matrices, STRU, ...), each record carrying a
      one-based STEP: N provenance header.
    • 2: per-step files routed into unified subdirectories (STRU/ for
      structures, MAT/ for sparse matrices such as dmr/tr/rr/dh/ds/lx, etc.),
      created on demand.
  • INPUT documentation updated to describe the fixed-name behavior of mode 1.

Output renames and provenance headers

  • Orbital information output renamed from Orbital to nao.txt.
  • Band-energy-term outputs renamed (kinetic_out -> eband_kin,
    vhartree_out -> eband_hart, vpp_local_out -> eband_vl,
    vpp_nonlocal_out -> eband_vnl, vxc -> eband_vxc), with per-ionic-step
    provenance headers added to band output.
  • Band-decomposed XC output renamed; LIBRPA producer reference files relocated
    accordingly (manifests updated).
  • LCAO k-space matrix output parameters renamed, and misleading docs fixed.
  • Text LCAO wavefunction output gains a provenance header; the integration
    comparison tool skips comment headers.
  • Ionic-step indices are printed one-based in user-facing outputs (STEP: 1,
    matching the g{N} filename suffix convention), including binary H(R)/S(R)
    and sparse DMR writers; governance doc updated accordingly.

MD / relaxation structure output

  • STRU output directory and naming unified between relax and MD.
  • MD STRU numbering now starts from 1, consistent with relax.
  • md_restartfreq replaced by out_freq_ion + out_stru.
  • Legacy STRU directory layout is preserved for MD structure writing, and the
    MD restart STRU lookup broken by the rework is restored.

Bug fixes

  • Fix MPI crashes in write_dipole when out_dipole=1.
  • Keep all ranks on the FFT path in write_elf (fixes ELF output hangs/crashes).
  • Reuse the fixed dH/dS CSR filename in append mode and log every write.
  • Terminate the DM(k) matrix body with a newline.
  • Route remaining PW outputs through out_app_flag and drop the incorrect
    LCAO-only availability claim.
  • Use the real out_app_flag value for PW wavefunction output; use local
    cached directory variables in ctrl_output_fp.

Tests

  • New three-mode unit tests covering out_app_flag naming and append
    semantics, save_mat overwrite, and write_hsk mode-2 rerun semantics.
  • Sparse-matrix reference files and DMR writer test updated for one-based
    STEP headers; DM(k) test crashes fixed.
  • Integration reference files updated for renamed outputs
    (props_cube.sh, props_mat.sh, LIBRPA manifests, etc.).

Backward compatibility

  • Default behavior (out_app_flag=0) is unchanged; mode 1/2 are opt-in.
  • md_restartfreq is removed in favor of out_freq_ion + out_stru;
    existing inputs using md_restartfreq must migrate.
  • Several output file names changed as listed above; downstream parsers and
    LIBRPA producer consumers should use the updated manifests.

Verification

  • Unit tests: new/updated tests under source/source_io/test,
    source/source_lr, and module test directories (three-mode naming/append,
    sparse writers, STEP headers).
  • Integration tests: updated references in tests/01_PW, 02_NAO_Gamma,
    03_NAO_multik, 08_RI, plus validation tools (props_*.sh,
    check_librpa_producer.py, compare_hsr_binary.py).

abacus_fixer added 8 commits October 16, 2026 14:47
…unified output directories

- Change out_app_flag from bool to int with three states:
  0: no append, no subdirectory (overwrite)
  1: append to same file (default, backward compatible)
  2: write to unified subdirectories
- Add new global directory variables in System_para:
  global_psi_dir, global_chg_dir, global_pot_dir,
  global_elf_dir, global_dos_dir, global_dip_dir, global_mat_dir
- Initialize directories in set_global_dir():
  out_app_flag==2 -> OUT.suffix/{PSI,CHG,POT,ELF,DOS,DIP,MAT}/
  otherwise -> all point to global_out_dir (backward compatible)
- Map legacy WFC->PSI, matrix->MAT
- Update all out_app_flag usages from bool to int semantics
- Broadcast new directory variables via Parallel_Common::bcast_string

Issue: deepmodeling#8151
- ctrl_output_fp.cpp: route chg/tau to global_chg_dir, pot/potes to
  global_pot_dir, ELF to global_elf_dir, dipole to global_dip_dir
- ctrl_output_pw.cpp: route wfc to global_psi_dir, dos to global_dos_dir,
  ldos to global_dos_dir
- ctrl_scf_lcao.cpp: route dos to global_dos_dir
- ctrl_runner_lcao.cpp: route ldos to global_dos_dir
- read_input.cpp: create subdirectories (PSI/CHG/POT/ELF/DOS/DIP/MAT)
  when out_app_flag==2
- Update docs: out_app_flag semantics from bool to int (0/1/2)

Issue: deepmodeling#8151
When out_app_flag defaults to 1 (not 2), all new unified directories
(PSI, CHG, POT, ELF, DOS, DIP, MAT) map to global_out_dir for
backward compatibility. Update io_sys_var_test.cpp expectations:
- global_matrix_dir: "OUT.test/matrix/" -> "OUT.test/"
- Add assertions for all 7 new directory variables
- read_input.cpp: change !out_app_flag to out_app_flag == 0 for
  matrix/WFC legacy directory decisions
- Update three unit tests to use int out_app_flag (0/1) instead of
  bool (false/true), matching the new function signatures:
  test_cal_dos.cpp, test_write_dos_pw.cpp, write_wfc_nao_test.cpp
- Update two comments to describe 0/1 semantics
Replace repeated PARAM.globalv.global_*_dir accesses with the local
const variables already declared at function top, eliminating the
net +4 PARAM global dependency introduced by the path routing change.
With out_app_flag changed from bool to int, the legacy WFC/ subdirectory
is no longer created when out_app_flag=0 (now maps to global_out_dir).
Update the multi-k wfc reference path accordingly.
- read_input.cpp: only create PSI/CHG/POT/ELF/DOS/DIP/MAT subdirectories
  when the corresponding output flag is enabled (out_app_flag=2)
- ctrl_scf_lcao.cpp: route write_dmk and angular momentum calculator
  output to global_mat_dir instead of global_out_dir
- hsr_legacy.cpp: route tr_nao.csr to global_matrix_dir
- pos_op_writer.cpp: route rr_nao.txt to global_matrix_dir
@mohanchen mohanchen added Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes labels Oct 16, 2026
@mohanchen

Copy link
Copy Markdown
Collaborator Author

@ABACUS-PR-BOT please help review the PR

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of 00ece15 with independent verification. Five implementation issues and three non-blocking suggestions are noted inline. Whitespace checks passed; the pinned governance check reported warnings only. No build or runtime tests were run, and CI diagnostics were unavailable.

abacus_fixer and others added 18 commits October 18, 2026 08:10
- Remove md_no_append ternary in hsr_legacy.cpp, pos_op_writer.cpp,
  hterm_writer.cpp, dhs_sparse_writer.cpp: use global_matrix_dir
  unconditionally (it already equals global_out_dir when
  out_app_flag != 2, so no behavior change for legacy modes)
- write_dmr: add dmr_dir parameter, route DM(R) to MAT instead of
  hardcoded global_out_dir; drop parameter.h include
- read_input.cpp: add out_dmr/out_mat_l/out_mat_dh/out_mat_ds to
  MAT subdirectory creation flags

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- global_stru_dir now follows out_app_flag: STRU/ for flag 2,
  global_out_dir otherwise (consistent with MAT/PSI/CHG pattern)
- relax_driver: route STRU_NOW/STRU{step}/STRU_FINAL through
  global_stru_dir instead of global_out_dir
- run_md: drop _MD suffix, use STRU{step} naming consistent with
  relax
- global_file: remove unconditional STRU dir creation from
  make_dir_out; remove global_stru_dir parameter
- read_input: create STRU dir on demand (out_stru>0 or MD)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- run_md: use out_freq_ion (with md_restartfreq fallback) as the
  effective ionic-step output frequency for both STRU and restart
- run_md: gate STRU output on out_stru > 0, consistent with relax
- read_input: simplify STRU dir creation to out_stru > 0
- read_inp_out: update out_stru description to cover MD
- read_input_item_md: mark md_restartfreq as deprecated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- commit 3a38485 made global_stru_dir follow out_app_flag (default 1
  maps it to the output dir), but the MD restart lookup still probed
  STRU_MD_{istep} there, so restart cases whose fixtures live in
  OUT.suffix/STRU/ (e.g. tests/04_FF/14_LJ_rescale_v) failed to start
- search the current global_stru_dir first, then fall back to the
  legacy OUT.suffix/STRU/ layout; accept both the new STRU{istep+1}
  naming (1-based, unified with relax) and the legacy STRU_MD_{istep}
- quit with a clear diagnostic when no restart structure file exists
- add unit test covering both naming schemes in a user read_file_dir
- add PSI/ candidate in read_wfc_nao so RT-TDDFT restart finds mode-2 snapshots
- route H(k) term outputs to MAT directory; add out_hsr_npz to MAT dir creation
  predicate; narrow DOS dir creation predicate to out_dos/out_ldos
- pass boolean append to save_mat in write_hk_common (was int out_app_flag)
- update out_app_flag/out_wfc_lcao INPUT metadata and position_matrix docs
- add mode-2 directory mapping and PSI layout roundtrip unit tests
Conflict resolutions:
- read_input_test.cpp: keep both added tests (OutAppFlagParsing ours,
  ExplicitNoccSurvivesUnresolvedElectronNumber upstream)
- relax_driver.cpp: identical condition; keep ours comment (md STRU
  output now handled by run_md.cpp, md_restartfreq removed)

Post-merge verification: make -j 30 in build_max_para_test, 100%
Built target abacus_max_para, 0 errors.
In append mode (out_app_flag=1), chg/tau/pot/potes/ELF cube outputs
now reuse a fixed filename (e.g. chgs1.cube, elftot.cube) so every
step overwrites the previous snapshot instead of accumulating
g{N}-suffixed files. Modes 0 and 2 keep the per-step g{N} suffix.
Add an explicit out_app_flag parameter to ModuleIO::write_dmk. In
append mode the geometry-step index is dropped from the filename
(istep_file = -1) so every ionic step overwrites the previous
snapshot, matching the fixed name that read_dmk already expects;
the file body still records the real ionic step.
Drop the g{N} suffix from tr_nao.csr and rr_nao.txt in append mode.
The filenames were changing every ionic step, which defeated the
existing MD append branches in save_sparse and the rr assemble path;
with fixed names those branches now accumulate all steps as designed.
Replace the hardcoded out_app_flag=0 in write_wfc_pw (a 2025-05-17
TODO) with an explicit parameter wired from inp.out_app_flag. In the
default append mode the PW wavefunction files now use fixed names
(wfk1_pw.dat), which finally matches what the restart reader
read_wf2rho_pw looks for; modes 0 and 2 keep the g{N} suffix.
write_chg_init/write_pot_init now keep the fixed _ini.cube names in
append mode so each snapshot overwrites the previous one, instead of
accumulating chgs1g{N}_ini.cube / potg{N}_ini.cube files. These
initial-density cubes are write-only snapshots; no reader consumes
them, so no restart path is affected.
Clarify that mode 1 reuses fixed filenames across steps (text records
append, cube snapshots overwrite), fix the remaining Boolean wording
in out_hsk, and note that out_chg=2/out_pot=3 initial-density cubes
omit the geometry-step index in append mode.
The fixed filename alone still truncated the file every ionic step.
Add a calculation parameter and open the stream with ios::app under
the same condition as open_sparse_file (calculation=="md" &&
out_app_flag==1 && istep>=0), so each MD step appends a new record
while other calculations start fresh.
Rule 22: user-facing output files print ionic steps starting from 1,
matching the g{N} filename suffix convention; zero-based counters such
as "STEP: 0" are not allowed.
Naming (mode 1 fixed name, modes 0/2 keep g{N}):
- DOS (LCAO+PW) and PDOS honor out_app_flag for the g{N} suffix
- lx/ly/lz matrices: Angmom_op::calculate takes out_app_flag

Append (relax now appends like MD; was calculation=="md" only):
- open_sparse_file, rr/dH/dS writers, DM(k): condition relaxed to
  (out_app_flag == 1) && (istep >= 0); SCF passes istep=-1, unaffected
- PDOS and lx/ly/lz append per ionic step in mode 1
- DM(k) drops the calculation parameter again (no longer used)

Step indexing (governance rule 22, one-based):
- STEP: headers of sparse text files and the binary CSR step field
  now start from 1 (hs_sparse_io, pos_op_writer, dhs_sparse_writer,
  hsr_writer)

Build: make -j 30 verified by developer.
Instead of numbered STRU{N} files, out_app_flag=1 now writes one
fixed-name STRU file that accumulates one structure block per written
step, for both relax and MD:

- unitcell::print_stru_file and relax_stru_io::write_stru take a new
  append flag; relax writes "# Ionic Step N" headers (one-based,
  governance rule 22) via the existing build_stru_header
- mdcell::print_stru_file takes frame_comment + append; MD frames are
  prefixed with "# MD step N". The MPI path broadcasts a base offset
  (lseek SEEK_END) and keeps absolute pwrite offsets; O_APPEND is
  deliberately avoided since Linux pwrite ignores the offset with it
- CIF output keeps single-frame semantics: fixed-name STRU.cif is
  overwritten each written step
- modes 0/2 keep the numbered STRU{N} files unchanged
- CLI help of out_stru/out_app_flag documents the mode-1 behavior

Build: make -j 30 verified by developer.
The DM(k) writer emitted the row newline only before each group of
elements, so the file ended without a trailing newline. In append mode
(out_app_flag == 1) the next " --- Ionic Step N ---" record was glued to
the last value of the previous step. Add a trailing newline before
closing the file and cover it with a regression assertion.
abacus_fixer and others added 9 commits October 31, 2026 20:33
…ails

The multi-k LCAO branch piped awk output straight into the comparison input
and ignored both the source-file existence and the awk exit status. A
missing wfk1g3_nao.txt/wfk2_nao.txt therefore left an empty *_mod.txt that
CompareFile.py accepted against a nonempty reference, recording a pass.
Check the source file, propagate the preprocessing failure, and only run
the comparison after successful preprocessing (as done for the gamma-only
branch).
Make out_app_flag a three-mode controller so the default (mode 0) is
disk-friendly: a fixed filename is truncated and overwritten on every
write, out_freq_ion only controls how often a write happens and never
how many files are kept.

Mode semantics:
- 0 (default): fixed filename, truncated and overwritten at every write
- 1: fixed filename, text records / cube frames are appended (first
  written step truncates so reruns do not keep stale frames)
- 2: subdirectory + per-step g{step} file (one-based step index)

Naming and open mode are decoupled across all writers: the g{step}
suffix is only inserted when out_app_flag == 2, and an explicit append
flag (true only for mode 1 and istep > 0) controls whether the file is
opened in append or truncate mode.

Covers cube/charge/potential/elf, DOS/PDOS, DM(R)/DM(k), H(R)/S(R),
H(k)/S(k), T/R/dH, position/angular-momentum operators, PW/LCAO wave
functions, and relax structure output.
Update unit tests to exercise mode 0 (fixed name, truncate/overwrite),
mode 1 (append), and mode 2 (per-step g{step} file) consistently.

- filename: g/e suffix only appears for out_app_flag == 2
- write_dmr / dmk: append flag passed explicitly, no default arg
- hsr/dhs/hsk writers: mode 0 overwrites, mode 1 appends, mode 2 per-step
- DOS/PDOS: mode 0 fixed name, mode 1 appends frames, mode 2 g-suffixed
- wfc nao read/write: read side only looks up g-suffixed names in mode 2
- io_sys_var: directory routing (subdirs only in mode 2)
- LCAO init_dm / output_hcontainer: pass append=false for one-shot CSR
…1/2 modes

Rewrite the out_app_flag description to document the three modes
(0 = fixed name overwritten, 1 = fixed name appended, 2 = per-step
g{step} file) and set the default to 0.

Update the descriptions of out_chg, out_pot, out_dmr, out_dmk, out_hsr,
out_hsk, out_wfc_lcao, out_wfc_pw, out_dos, out_mat_l, out_stru, and
out_elf so that the g{step} suffix is described as mode-2-only, and
modes 0/1 keep the fixed filename (truncate vs append).
Update the user-facing documentation so the g{step} suffix is described
as out_app_flag == 2 only, and modes 0 (truncate/overwrite) and 1
(append) keep the fixed filename.

- output-specification.md: cube/DOS/ELF naming rules and example tables
- hs_matrix.md: H(R)/S(R) and H(k)/S(k) file naming per mode
- wfc.md: PW g*/e* suffixes only in mode 2; modes 0/1 reuse fixed names
- dos.md: doss{spin}.txt / pdoss{spin}.txt default, g-suffix in mode 2
- density_matrix.md: DM(k) default names without g, mode 2 adds g{step}
- position_matrix.md: rr_nao.txt vs rrg{step}_nao.txt and MAT/ subdir
…suffix

The cube validation script now expects g-suffixed cube files only when
out_app_flag == 2; with mode 0 the fixed-name cube files are reused.
…tep for PW wfc

In out_app_flag == 1 the fixed-name wavefunction file accumulates one
frame per ionic step. Previously, with out_freq_ion > 0 and a small
out_freq_elec, every electronic iteration within a step targeted the same
file: at istep 0 each iteration truncated (only the last survived), while
at istep > 0 each iteration appended another frame. This produced an
inconsistent trajectory (one frame for step 0, N frames for later steps).

Now in mode 1 ctrl_iter_pw ignores out_freq_elec and only writes the
last electronic iteration (iter == scf_nmax, converged, or final_output)
of each ionic step selected by out_freq_ion, so the appended file holds
exactly one frame per step. Modes 0 and 2 keep the out_freq_elec
intermediate-iteration behavior unchanged.

LCAO wfc output is unaffected because ctrl_scf_lcao runs once per ionic
step (after SCF) and does not gate on out_freq_elec.

Note: this selection logic lives inside the PW SCF loop and is exercised
by integration tests rather than isolated unit tests.
Sync local branch with the online PR deepmodeling#8159 head, which merged the latest
upstream/develop. Local out_app_flag mode-0 rework (17 commits) is preserved.

Key develop updates brought in:
- Move Structure_Factor, Soc, wf_op kernels to source_cell
- relax_driver / relax_nsync / relax_sync refactors
- TDDFT boundary-shift propagation restore
- OFDFT TF weight fix
- DFPT nlcc core-response terms
- Various build/test fixes

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head has two build integration mismatches in the INPUT documentation target and relaxation-driver test, detailed below.

Integrating newer develop 9d99f20e24c2efc8c56d68104bcff0b21fb912ac also adds an out_sol call to write_vdata_palgrid that omits the PR's required append argument. In that hypothetical combination, io_basic needs the named append_cube argument, aligned out_sol help, and explicit final-frame selection for the new solvent relaxation metric. This third mismatch exists in the prospective merge.

Verification: static source/link-closure review of head 386e195aa3de24acb1d4dced220500051e419b67 against API base 0ae4d926b90e53f3679e5f71e69b0dd804b8cfad, plus independent source review of its hypothetical merge with the newer develop commit (tree 31d00737edc3627827b510b972d6fbe6218e1f4a). No builds or tests were executed.

Comment thread source/source_relax/relax_stru_io.h Resolved
@ABACUS-PR-BOT

Copy link
Copy Markdown
Collaborator

Integrated develop 9d99f20e24c2efc8c56d68104bcff0b21fb912ac in 7811506452634edbda1741b4a222eb18899bd76d and fixed the three reported build integration mismatches. The solvent writer now passes append_cube, its help matches the output policy, and the relaxation validator selects the final cube frame (271.0889616), rather than the first (247.6910838).

Verification passed on this candidate: serial PW YAML and io_basic; MPI/LCAO main and relaxation-driver tests; nine parser/CLI/shell tests; both existing solvent cases on two ranks; six mode/cadence combinations and disabled controls; and the three-rank empty-partition test. Generated docs, --check-input, and actual help checks also passed. Numerical references and tolerances are unchanged. CI verification remains unavailable; deferred restart issues remain open.

abacus_fixer added 14 commits November 1, 2026 14:26
When out_freq_ion > 0, md_restartfreq was silently ignored because
mdrun->write_restart lived inside the do_output_step block gated by
eff_output_freq (= out_freq_ion). Restart files therefore followed the
STRU output cadence instead of md_restartfreq.

Move write_restart into its own block gated by md_restartfreq so the two
cadences stay independent. Extract Run_MD::is_cadence_step to make the
predicate unit-testable, and add focused tests covering disabled
frequency, per-freq firing, and the independence regression guard.

Also correct the md_restartfreq INPUT description: it controls
Restart_md.txt cadence (independent of out_freq_ion), not a deprecated
output-frequency alias.
The relax_stru_io::write_stru mock in test_relax_driver.cpp declared
nine parameters while the header (relax_stru_io.h) declares ten (the
trailing `const bool append`). relax_driver.cpp calls the ten-argument
overload, so the linker could not resolve write_stru and
MODULE_RELAX_relax_driver_test failed to build.

Add the missing tenth bool parameter to the mock.
Describe the out_freq_ion cadence and per-mode file layout actually used
by MD, and stop claiming that MD structure output shares the restart-file
frequency. Note that MD writes neither STRU_NOW nor STRU_FINAL.
The MD structure-frame cadence follows out_freq_ion while Restart_md.txt
follows md_restartfreq, so a checkpoint only has a restorable frame when
the two cadences coincide (and out_stru is positive). Document this
constraint in the md_restart help instead of implying that any checkpoint
can always be read back.

Restart rework is intentionally out of scope here and is deferred to a
follow-up:
- write a dedicated single-frame restart snapshot from run_md, then drop
  find_md_restart_stru/stru_frame_io and the legacy STRU/ stat in run_md;
- stop --check-input from writing STRU_restart_frame during input parsing;
- remove the io_input -> source_cell edge and the stru_frame_io.cpp entries
  in the io test CMakeLists (and Makefile.Objects);
- keep frame and checkpoint aligned (the current fallback can resume from
  an older frame) and validate out_stru=0 for MD restart.
The first int of the native binary CSR record changed meaning from a
zero-based ionic step (v3.11.0-beta9/beta10) to a one-based step without
leaving any marker, so external readers silently misread the step and
shifted every following field. Prefix each record with the magic
0x48535242 ('HSRB') and a format version (2), which keeps the one-based
step while making the layout self-identifying and version-checkable.

Readers now reject a record whose first int is not the magic instead of
guessing the layout. Update the integration comparison tool, the CSR
writer unit tests, the out_hsr help text, and the format documentation
with the migration note.

Verification:
- cmake --build build --target MODULE_IO_hsr_writer_test
- OMP_NUM_THREADS=1 ctest --test-dir build -R MODULE_IO_hsr_writer (2/2 passed)
- cmake --build build --target io_input (compiles read_inp_out.cpp)
- python3 -m py_compile tests/integrate/validation_tools/compare_hsr_binary.py
Mode 1 accumulated one frame per written step, but several writers decided
the append state locally: the relax STRU writer always appended, and the
DM(R)/H(R)/dH callers passed append=true unconditionally and relied on the
leaf writer to re-apply the istep > 0 guard.

Compute the append state at each call site as
out_app_flag == 1 && istep > 0 so a fresh run truncates the fixed-name file
at step 0 and never stacks frames from a previous run, matching the cube,
DM(k) and DOS writers.
The provenance step used by after_all_runners was written from both
before_scf and the TDDFT runner. Record it in the three-argument
before_scf overload instead, so every LCAO entry path funnels through
one write while still passing the output index explicitly.
…iters

write_bands, write_wfc_nao and vxc_op_tools each carried a local copy of
the version/timestamp header builder. Both lines now come from
ModuleIO::provenance_header() and ModuleIO::local_timestamp() in
module_output/provenance_header.h; each writer keeps only its own step
line. local_time_stamp() is removed in favor of local_timestamp().
Header text is unchanged, keeping bands*.txt, wfk*_nao.txt and
eband_*.txt formats compatible.
Replace the hand-maintained `out_app_flag == 2` subdir predicate in
`ReadInput::create_directory` with a lazy `ModuleBase::ensure_directory`
call at each output site. The old predicate had to enumerate every
writer's flags by hand and was patched in three separate review rounds;
moving creation to the writers removes that whole class of drift.

Each ensure sits inside the existing output flag guard, so only
directories that actually receive a file are created. Writers that read
`PARAM.globalv.*_dir` directly (write_dh_perI, write_wfc_nao,
output_mat_sparse) ensure in the leaf; dispatcher-level sites hoist the
directory into a local so the ensure and the write share one lookup.
read_set_globalv.cpp calls stru_frame_io::extract_last_frame through the
io_input object library, but abacus_input_yaml listed neither the source
nor the cell library, so the documentation executable failed to link.
Add the source file to that target, matching the INPUT test targets.
…olation

The boundary-shift propagation restore (6cf34b9) derives
positions_updated from the global MD step (step_ + step_rst_), so a
restarted run's first force evaluation wrongly marks positions as
updated and immediately advances the charge-extrapolation history
with a degenerate state, shifting the PW md_nvt trajectory.

Pass an explicit positions_updated flag into MD_func::force_virial
instead: false for the run's initial evaluation (including restarts),
true after the first MD drift. Add a restart regression unit test and
regenerate tests/01_PW/md_nvt/result.ref.
Resolve the add/add conflict in read_input_item_test.cpp: both sides
appended new InputTest cases at the same location. Keep the branch's
OutMatComponentPrecision / OutStruCifRejectedForMd tests and the
upstream DfptMixType / DfptKerkerA2 / DfptBroydenNdim domain tests.
The out_app_flag 0/1/2 rework made mode 0 (overwrite fixed filename)
the default, so the two MD cases without an explicit out_app_flag now
keep only the last ionic step in hk_nao.txt / hks1_nao.txt, while
their references hold two frames. Restore the historical append
behavior explicitly to keep the multi-step accumulation coverage.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static incremental review at ca7db6e2 found a new MPI STRU write-offset race. The feedback also covers remaining output-retention and metadata gaps, plus newly added test fixtures. No builds or runtime tests were run.

// broadcasting the MPI_Offset held by rank 0: after the barrier the header
// is in place, so the end of the file is the current base offset plus the
// header that was just written (zero for a truncated fresh file).
const off_t file_end = lseek(file, 0, SEEK_END);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Rank 0 can write the first species header before another rank executes this SEEK_END. That rank then observes a larger EOF, so atom records in the same frame use different base offsets and the STRU can be corrupted, even in a fresh MD run. Share one offset through a guarded communicator wrapper, or synchronize all seeks before the first write, and cover the ordering with an MPI round-trip test.

hamilt::folding_HR(*hR, hk.data(), params.kv->kvec_d[ik], nlocal, 0);
#endif
std::string fk = global_out_dir + kprefix + tag;
std::string fk = mat_dir + kprefix + tag;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: these dense dH(k) filenames omit the ionic step, while mode 2 disables append, so successive written geometries overwrite the same files. Include g{step} in mode 2 or document the dense-output exception. This retained PR behavior also exists at the previous reviewed head.

kv.isk.resize(1);
kv.isk[0] = 0;

const std::string out_dir = "./";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the newly added DMK mode tests write and remove fixed filenames in the working directory; the new DOS mode fixture has the same pattern. Use fixture-specific paths under testing::TempDir() for these additions and reuse them for writes, reads and cleanup, as required by rule 17.

abacus_fixer added 2 commits November 2, 2026 19:18
Recent bug fixes shifted the converged energies of the two remaining
active LCAO cases; 17_DS_DFTU has never been wired into CI/CD, so the
references are updated manually for now. The folder will be moved and
integrated elsewhere later.

- CASES_CPU.txt: keep only 01_LCAO_SPIN_S2_Z and 03_LCAO_DFTU_S2_Z
- INPUT: tighten scf_thr from 1e-6 to 1e-8 in both cases
- result.ref: refresh etot/etotperatom (shifts of ~1.6e-5 eV and
  ~1.0e-4 eV) generated with the tightened threshold

Verified: Autotest.sh -a build_max_para_test/abacus_max_para -n 4 -o 1
passes both cases (v3.11.0-beta12, ca7db6e).
The MD restart inputs (Restart_md.txt, STRU_MD_10) were force-tracked
inside OUT.autotest/, where test-cleanup scripts delete OUT.* without
distinguishing tracked inputs from run artifacts.

- git mv both files to the case directory top level
- INPUT: set read_file_dir ./ so the restart reader looks up
  Restart_md.txt and STRU_MD_{step} directly in the case directory
  (user-provided read_file_dir bypasses the OUT.{suffix}/STRU search)

Verified: Autotest.sh -a build_max_para_test/abacus_max_para -n 4 -o 1
-r 14_LJ_rescale_v passes all 4 checks (etot/etotperatom/force/stress);
energies identical to the reference, confirming the restart from step 10.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes Solve before 3.11

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants