feat: support LMDB datasets in dp eval-desc and dp embed #6034

Summary

dp train and dp test read deepmd/lmdb datasets natively since 3.2 (LmdbDataset, LmdbTestData), but dp eval-desc and dp embed still build on expand_sys_str + DeepmdData, neither of which has any LMDB handling. Passing an LMDB dataset therefore fails immediately with RuntimeError: Did not find valid system, even though the same file is accepted by dp test.

Versions

• deepmd-kit 3.2.0 (pip wheel, --pt backend) — reproduced
• also reproduced on a 3.2.0 dev build (3.2.0b1.dev67+g73de44b1f)
• dpdata 1.1.0, Python 3.12

Minimal reproduction

data.lmdb is a directory containing data.mdb (a dpdata deepmd/lmdb dataset, one or more composition groups):

dp --pt test      -m model.pt -s data.lmdb -n 0         # works (LmdbTestData, nloc groups)
dp --pt eval-desc -m model.pt -s data.lmdb -o desc      # RuntimeError: Did not find valid system
dp --pt embed     -m model.pt -s data.lmdb -o emb.hdf5  # RuntimeError: Did not find valid system

The underlying calls that fail:

from deepmd.common import expand_sys_str
from deepmd.utils.data import DeepmdData

expand_sys_str("data.lmdb")                         # -> []
DeepmdData("data.lmdb", type_map=["Fe", "O", "C"])  # -> FileNotFoundError: No set.* is found

Root cause

• deepmd/entrypoints/eval_desc.py: all_sys = expand_sys_str(system) → if len(all_sys) == 0: raise RuntimeError("Did not find valid system"), then DeepmdData(system_path, set_prefix="set", ...)
• deepmd/entrypoints/embedding.py: same pattern
• deepmd/utils/data.py (DeepmdData) has no LMDB branch
• Compare deepmd/entrypoints/test.py, which dispatches on is_lmdb() to LmdbTestData

Checked on 3.2.0 GA, a 3.2.0 dev build, and master: lmdb appears 0 times in entrypoints/eval_desc.py, entrypoints/embedding.py, and utils/data.py (utils/data_system.py only special-cases LMDB in process_systems).

Impact

Any wrapper that mirrors the CLI has to keep an npy/mixed duplicate of a dataset that dp train/dp test already consume from a single LMDB file. LMDB is currently the format that makes large validation sets practical to ship and migrate, so the asymmetry is user-visible; the failure is also only discoverable after a job has started.

Proposal

Route LMDB through the same reader used by dp test (or a shared data-source iterator) inside eval_desc/embedding, with an explicit answer for the output layout:

1. Ordering — LMDB rows follow evaluator order (LmdbTestData iterates nloc groups in ascending order, storage order inside a group). The descriptor/embedding arrays and the HDF5 dataset need a documented per-frame ordering, plus a note that one LMDB is reported as a single data source.
2. Group naming — for npy the outputs are keyed by system directory; an LMDB has no directory-level system names, so a documented rule (e.g. one output group per frame_system_ids entry, or <lmdb name> [nloc=N]) would avoid a silent semantic change for consumers that key on system names.

Documenting explicitly that eval-desc/embed currently require deepmd/npy would already help users; the code path otherwise only fails at runtime.

Happy to test a patch, or to provide a small LMDB fixture and an equivalent npy counterpart for a regression test if that is useful.