Skip to content

fix: keep dash-named parameters in netcdf round trip - #1025

Open
MaykThewessen wants to merge 1 commit into
PyPSA:masterfrom
MaykThewessen:fix/netcdf-dash-parameters
Open

MaykThewessen wants to merge 1 commit into
PyPSA:masterfrom
MaykThewessen:fix/netcdf-dash-parameters

Conversation

@MaykThewessen

Copy link
Copy Markdown
Contributor

Note

AI-assisted. Claude wrote the fix and tests, and ran the checks listed below. I reviewed the diff.

Changes proposed in this Pull Request

read_netcdf silently drops a parameter whose name contains a dash: after m.parameters["my-param"] = ..., a round trip through netcdf returns the model without it. read_netcdf finds the container of each data variable and attribute by splitting its key at the last dash. That works for variables-x-var-labels, where the last part is a fixed field name, but parameters are stored as parameters-<name>, so parameters-my-param ends up under parameters-my. Attribute keys have the same problem, which also dropped parameter attributes with a dash and the _multiindex attribute of a dimension with a dash in its name (legacy semantics), so that dimension came back without its MultiIndex.

Each key now belongs to the longest known container prefix, taken from the container names (plus objective and parameters).

Overlap with #1021 and with #1024: all three touch read_netcdf in linopy/io.py and merge without textual conflicts. The aux-coords branch derives its own container prefixes by splitting at the last dash, so with both merged as they are, a dash-named parameter yields a bogus prefix parameters-my that takes the scalar coords of parameters starting with my-. Whichever lands second should drop its prefixes/scalar_owner and use owner() from this PR (see details).

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included.
  • I consent to the release of this PR's code under the MIT license.
What was checked (AI-generated)
  • New test_model_to_netcdf_with_dash_named_parameters in test/test_io.py: the model fixture plus parameters["my-param"] over a dimension my-dim, and a parameter attribute my-attr. Asserts assert_model_equal (compares parameters) and the parameter attributes. Fails on master's io.py under legacy and v1, passes with the fix.
  • New test_model_to_netcdf_with_dash_named_multiindex_dim (legacy only, v1 rejects MultiIndex): a variable and a parameter over a MultiIndex dimension multi-dim. Fails on master's io.py (the MultiIndex is lost for both), passes with the fix.
  • Other users of the last-dash split checked: data variable fields of all containers (labels, lower, upper, coeffs, vars, const, sign, rhs, the CSR fields such as indptr, _active_positions) and linopy's own attribute keys contain no dash, so container_names and the variable, constraint, expression and objective paths are unaffected by names with dashes. The existing test_model_to_netcdf_with_dash_names (x-var next to x-var-2) covers the longest-prefix choice.
  • Not fixable on read: variable x with a dimension mi-dim and a variable x-mi with a dimension dim both write the attribute variables-x-mi-dim_multiindex. Read assigns it to x-mi. Needs both names in one model; left as is.
  • test/test_io.py, test/test_csr.py and the other netcdf users (test_sos_reformulation, test_indicator_constraints, test_dtypes, test_scaling, test_piecewise_constraints, test_fix_relax): 1865 passed. The 6 failures are TestSignParameter SOS2 cases in test_piecewise_constraints.py (HiGHS here has no SOS support) and fail identically on master.
  • ruff check, ruff format --check clean; mypy reports no errors in the changed files (2 existing ones in solvers.py and remote/oetc.py).
  • Merged with fix: keep auxiliary coordinates of dense containers in netcdf round trip #1024 (dce6104): no textual conflicts, its tests and these pass, but a probe with parameters["my-param"] next to a parameter carrying a scalar coord my-scalar loses my-scalar on read. Replacing its prefixes/scalar_owner block with ds[c].ndim == 0 and owner(str(c)) != prefix in get_prefix fixes that; with it, test/test_io.py and test/test_csr.py pass (1033 passed). With both, the aux coord of a dash-named parameter also comes back.
  • Merged with fix: write timezone-aware coordinates in Model.to_netcdf #1021 (fix/netcdf-tz-aware-coords): no textual conflicts, its tz tests and these pass (25 passed, 3 skipped for -k "tz or dash"), and a dash-named parameter and variable over a tz-aware dimension my-time come back with Europe/Amsterdam.

🤖 Generated with Claude Code

…ad_netcdf

read_netcdf assigned each data variable and attribute to its container
by splitting the key at its last dash. Parameters are stored as
parameters-<name>, so a parameter named my-param was assigned to
parameters-my and dropped; the same happened to parameter attributes
and _multiindex attributes of dimensions with a dash in their name.
Keys now belong to the longest known container prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codspeed

codspeed Bot commented Oct 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing MaykThewessen:fix/netcdf-dash-parameters (9cc45e7) with master (1d3c516)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant