Charge-density restarts: G-space truncation and different SCF convergence with binary and cube files #7999

Details

I recently read Chen et al.'s paper, Evaluating Predicted Densities, Hamiltonians, and Density Matrices as Periodic SCF Initializers, and became interested in an observation in Table 3: restarting from the converged charge density gives a median of 3 SCF iterations, whereas restarting from the converged density matrix or Hamiltonian gives a median of 1 iteration.

atomic_pair_20260920T103613Z_01-03.zip

After inspecting the ABACUS source and running controlled tests on the authors' agm001003669 example (see above), I found strong evidence that G-space truncation in the standard binary charge-density restart is the main reason for the additional iterations in this case. I would like to discuss the intended roles of the binary and cube restart paths, potential user confusion, and possible improvements.

Reproduction

• ABACUS v3.11.0-beta9, commit 239e8f43e93f720701beee1f3184af74426fb97e; no C++ modifications.
• LCAO/PBE, ecutwfc = 120 Ry, ecutrho = 480 Ry, scf_thr = 1e-7, Broyden mixing, mixing_beta = 0.2.
• FFT grid: 45 × 45 × 108; parallel configuration: 8 MPI processes × 1 OpenMP thread.
• The same executable, pseudopotentials, orbitals, and SCF settings were used throughout.

First, I performed a complete SCF calculation from an atomic initial guess, using out_chg 1 17 to save both the binary restart and a high-precision cube file. I then ran two separate restarts using these outputs:

init_chg      file
read_file_dir ./INIT
out_chg       1 17

The two restart INPUT files were identical. The binary test's INIT directory contained only {suffix}-CHARGE-DENSITY.restart; the cube test's INIT directory contained only chg.cube. The logs confirmed that each run read the intended file.

The cube density values were written in scientific notation with 17 digits after the decimal point. The cube produced after the one-step cube restart was byte-for-byte identical to the reference cube.

Source inspection and numerical checks

In the LCAO path inspected, both outputs originate from the same rho_save, but they preserve different representations:

• Binary: rho_save is transformed to reciprocal space, and only the G components retained by the charge-density plane-wave basis are written. This cutoff-limited set does not include all Fourier components representable on the full FFT grid.
• Cube: rho_save is written directly on the full real-space grid.

Relevant code: binary output, G-vector selection, and cube output.

Transforming the full cube density to reciprocal space, retaining only the G set present in the binary file, and transforming back introduces a normalized L1 density difference of approximately 1.38401e-5. This is close to the binary restart's first-iteration DRHO of 1.38407e-5. Meanwhile, the maximum difference between the binary coefficients and the cube's Fourier coefficients on the retained G set is only approximately 1.4e-17.

These results strongly support loss of high-G components, rather than floating-point precision of the stored coefficients, as the dominant cause in this example. I have not verified this explanation across the paper's entire benchmark dataset.

Why the current behavior may be confusing

With init_chg=file, ABACUS attempts to read the binary restart first and falls back to cube if that fails. Consequently, when both files are present in read_file_dir, the binary file takes precedence even if the user explicitly requested a high-precision cube. See the file-selection logic.

Although the selected file is reported in the log, users may reasonably assume that these are simply two storage formats for the same converged density. In practice, they can preserve different information and produce different SCF restart behavior. For oracle tests or evaluations of AI-predicted density initializers, this representation loss can also confound the assessment of initializer quality.

Questions for discussion

1. What are the intended use cases for maintaining both reciprocal-space binary and real-space cube charge-density restarts? Are there performance, compatibility, or historical considerations behind the G truncation and binary-first selection policy?
2. Could the documentation and initialization logs explain this distinction more explicitly? Would an explicit input option for choosing the restart format be useful, so users do not need to move files to control which one is read?
3. Would it be worth supporting a binary format that preserves the complete real-space grid density, or another mode that allows a numerically faithful round trip without additional G-space truncation?

I would appreciate the maintainers' perspective on the current design and whether changes to the interface or default behavior would be appropriate.

Have you read FAQ on the online manual http://abacus.deepmodeling.com/en/latest/community/faq.html

• Yes, I have read the FAQ part on online manual.

Task list for Issue attackers (only for developers)

• Understand the problem or question described by the user.
• Check if the issue is a known problem or has been addressed in the documentation.
• Test the issue or problem on a similar system or environment, if possible.
• Identify the root cause or provide clarification on the user's question.
• Provide a step-by-step guide, including any necessary resources, to resolve the issue or answer the question.
• If the issue is related to documentation, update the documentation to prevent future confusion (optional).
• If the issue is related to code, consider implementing a fix or improvement (optional).
• Review and incorporate any relevant feedback from users or developers.
• Ensure the user's issue is resolved or their question is answered and close the ticket.