[tlmquintino]

This PR lets statistics-mtg2 accumulate the IFS accumulated parameters from forecast start. It is needed by models that send per-step (e.g. hourly) increments, such as pmap-ifs. It has two commits.

Mappings (2fac5fce)

The PR adds 19 identity accumulate entries (typeOfStatisticalProcessing = 1, paramIn == paramOut) to share/multio/param-mappings/statistics_param_mappings.yaml:

group	paramIds	units
water	228228, 228143, 3062, 228144, 260259, 231002, 231010, 231012	kg m**-2
energy	146, 147, 176, 177, 169, 175, 178, 179, 212	J m**-2
stress	180, 181	N m**-2 s

• With the entries, a client that sends increments under the accumulation's own paramId can keep strict mapping on. Without them the action throws Mapping for param=228143 and typeOfStatisticalProcessing=1 is undefined!.
• No unit conversion is done: the increments must already be in the units of the paramId.
• Every id was checked against the parameter database; each is an accumulation in the units above.
• The generator does not produce these entries, so they are appended as manually added mappings.

Option window-origin (8e846118)

The new statistics-mtg2 option window-origin takes window-start (the default, unchanged) or forecast-start.

With forecast-start, the statistic accumulates from the simulation start (the first-step flush) and is never reset. output-frequency becomes the cadence at which the running value is emitted, with:

• step = the current step;
• timespan = fs;
• the elapsed length as misc-distanceFromPreviousStepInSeconds.

A model sending hourly increments therefore gets 0-3, 0-6, 0-9 h accumulations at every output step. It needs no last-step flush per output step and does not need to know the run length.

• A sample that lands on an emission point is emitted at once.
• A last-step flush emits only the open tail.
• Completeness is judged against the elapsed span.

Supported combinations are accumulate/average/maximum/minimum, an hourly output-frequency, forward-offset windows and instantaneous input. Other combinations are rejected when the action is built.

The restart format is unchanged, since the window state was already serialized, and a restored window keeps its original forecast start.

Why not the existing options. The 2.11.2 workaround was a run-long window plus a last-step flush at every output step. It fails on develop, because an open window is now emitted with its nominal length (step=3, timespan=6h gives "forecastTime must be non-negative").

Tests

• tests/multio/action/statistics-mtg2/ForecastStartWindow.cc, 13 cases:
  • emissions at 3, 6 and 9 h, with the correct sums and elapsed length;
  • tail, gap and restart;
  • rejected configurations, and the default unchanged.
• tests/multio/action/statistics-mtg2/ParamMapping.cc: the 19 entries with strict mapping on, and rejection for other operations.
• statistics / encode / mapping ctest subset: 182/182.
• End to end with pymultio 2.11.2.27 and a real local FDB: 7/7 of pmap-ifs's from-start accumulation tests pass with strict mapping.
• Full ctest: 225/226. test_multio_subtoc fails with subtocExists() in an fdb5-only plan with no statistics action; I have not yet checked whether it also fails on the unmodified develop tip.

Questions for reviewers

• A restart that finds no state for a from-start field currently starts a new accumulation at the restart step, with an elapsed-duration timespan. Should this be an error instead?
• Should a last-step flush emit the open tail of a run that ends between outputs? It does now.
• Should the option extend to stddev and to calendar periods?

🌈🌦️📖🚧 Documentation 🚧📖🌦️🌈
https://sites.ecmwf.int/docs/dev-section/multio/pull-requests/PR-311

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.55914% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 31.04%. Comparing base (afd32cf) to head (8e84611).

Files with missing lines	Patch %	Lines
...c/multio/action/statistics-mtg2/OperationWindow.cc	74.19%	8 Missing ⚠️
src/multio/action/statistics-mtg2/Statistics.cc	93.22%	4 Missing ⚠️
...ltio/action/statistics-mtg2/ForecastStartWindow.cc	98.95%	3 Missing ⚠️
...ests/multio/action/statistics-mtg2/ParamMapping.cc	98.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #311      +/-   ##
===========================================
+ Coverage    30.26%   31.04%   +0.78%     
===========================================
  Files          410      411       +1     
  Lines        42236    42686     +450     
  Branches      2062     2099      +37     
===========================================
+ Hits         12783    13254     +471     
+ Misses       29453    29432      -21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.