Rust docs examples by Choochmeque · Pull Request #55 · ecmwf/odc

Choochmeque · 2026-09-24T12:59:05Z

Description

Rust documentation and examples

Contributor Declaration

By opening this pull request, I affirm the following:

All authors agree to the Contributor License Agreement.
The code follows the project's coding standards.
I have performed self-review and added comments where needed.
I have added or updated tests to verify that my changes are effective and functional.
I have run all existing tests and confirmed they pass.

📖 Documentation 📖
https://sites.ecmwf.int/docs/dev-section/odc/pull-requests/PR-55

codecov-commenter · 2026-09-24T13:04:44Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.03%. Comparing base (c94e0fb) to head (d76e00e).
⚠️ Report is 20 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop      #55      +/-   ##
===========================================
+ Coverage    87.00%   87.03%   +0.02%     
===========================================
  Files          189      189              
  Lines        15125    15219      +94     
  Branches      1370     1370              
===========================================
+ Hits         13160    13246      +86     
- Misses        1965     1973       +8

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

simondsmart

Generally good. You need to include the stride behaviour in the custom encoders/decoders.

I also think that the Span functionality is the perfect place for a rust enum. Consider that the span is essentially a map<column-name, enum<integers, strings, floats>>

simondsmart · 2026-10-01T12:30:20Z

+    uint64_t offset() const;
+    uint64_t length() const;
+    bool equals(const SpanWrapper& other) const;
+    rust::Vec<int64_t> integer_values(rust::Str column) const;


This misses something of the underlying functionality of Span - and that can be seen in the documentation example.

This is important in a context where you may not know what the types you are going to have - or indeed, which columns are going to be there in the first place.

Fixed in 652593c

simondsmart · 2026-10-01T12:32:11Z

+
+      .. code-block:: rust
+
+         // Decode all columns into a Polars DataFrame


This Polars stuff needs a bit more upfront commentary. Because this is different to the model for the C and Fortran data (and is more similar to the python one).

Fixed in 1a8b20e

simondsmart · 2026-10-01T12:35:36Z

+
+         .. note::
+
+            Rust interface does not support data encoding from a column-major layout. In this case, encode from a Polars ``DataFrame`` or from a :ref:`custom memory layout <encoder-custom-layout>` instead.


Is there a particualr reason why not?

Fixed in bc2daa2

simondsmart · 2026-10-01T12:36:46Z

+            // set up the data here...
+
+            let columns = [
+                RawColumn {


Do we support controlling the data stride?

Fixed in bc2daa2

simondsmart · 2026-10-01T12:38:28Z

+
+         let options = WriteOptions::default();
+
+         // Or encode into an open eckit DataHandle


I want to imagine writing to a more general writable rust object (imagine a network socket, or something else). Can we show the eckit behaviour of capturing a more general rust writable object?

Fixed in d76e00e, requires ecmwf/eckit#375 (DataHandle::from_writer)

Choochmeque added 14 commits September 25, 2026 12:04

Add Rust tab with odc_ls example to data listing docs

d52075c

Add Rust tabs with header and custom encoding examples to docs

8f56b4e

Add row-major encoding API with Rust docs example

7a53321

Bridge the Span API with an archival index example

d92eb3b

Serialize encodes until codec optimizer statics are initialized

fa7fa0c

Pin integer behaviour on the Frame::span entry point

eac7620

Add Rust tab to the first steps guide

6f41aeb

Add Rust tabs to the decoding data guide

aa33a31

Add Rust tabs to the encoding data guide

9d1e9cf

Add Rust interface section to the API design guide

44cda8f

Add Rust reference page

6a994e0

Mention Rust in the introduction and README

ffa1ccc

Exclude .venv from Sphinx sources

d2158ab

Add Rust interface section to the development guide

b17a9f2

Choochmeque force-pushed the rust-docs-examples branch from 9f5f71c to b17a9f2 Compare September 25, 2026 10:04

Choochmeque added 3 commits September 25, 2026 12:13

Add Rust section to the installation guide

454b233

Drop the pyodc comparison from the Rust interface section

cc457a8

Clean up documentation

ad32025

Choochmeque force-pushed the rust-docs-examples branch from bbde5a8 to ad32025 Compare September 25, 2026 10:21

Choochmeque requested a review from simondsmart September 30, 2026 17:45

simondsmart requested changes Oct 1, 2026

View reviewed changes

Choochmeque added 5 commits October 1, 2026 14:49

Describe the Rust decoding model in the decoder section

1a8b20e

Enumerate span columns as typed value sets

652593c

Update eckit and eckit-sys dependencies to use the develop branch

3009bef

Add strided cell layouts for encoding and decoding

bc2daa2

Encode into Rust writers through eckit data handles

d76e00e

Choochmeque requested a review from simondsmart October 12, 2026 14:02

simondsmart merged commit e766a08 into develop Oct 15, 2026
195 of 218 checks passed

Choochmeque deleted the rust-docs-examples branch October 16, 2026 15:43

Navigation Menu

Repository navigation

Rust docs examples - #55

Rust docs examples#55
simondsmart merged 22 commits into
developfrom
rust-docs-examples

Choochmeque commented Sep 24, 2026 •

edited by github-actions Bot

Loading

Uh oh!

codecov-commenter commented Sep 24, 2026 •

edited

Loading

Uh oh!

simondsmart left a comment

Uh oh!

simondsmart Oct 1, 2026

Uh oh!

Choochmeque Oct 9, 2026

Uh oh!

simondsmart Oct 1, 2026

Uh oh!

Choochmeque Oct 9, 2026

Uh oh!

simondsmart Oct 1, 2026

Uh oh!

Choochmeque Oct 9, 2026

Uh oh!

simondsmart Oct 1, 2026

Uh oh!

Choochmeque Oct 9, 2026

Uh oh!

simondsmart Oct 1, 2026

Uh oh!

Choochmeque Oct 9, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		.. code-block:: rust

		// Decode all columns into a Polars DataFrame


		.. note::

		Rust interface does not support data encoding from a column-major layout. In this case, encode from a Polars ``DataFrame`` or from a :ref:`custom memory layout <encoder-custom-layout>` instead.


		let options = WriteOptions::default();

		// Or encode into an open eckit DataHandle

Conversation

Choochmeque commented Sep 24, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Contributor Declaration

Uh oh!

codecov-commenter commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

simondsmart left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Choochmeque commented Sep 24, 2026 •

edited by github-actions Bot

Loading

codecov-commenter commented Sep 24, 2026 •

edited

Loading