Skip to content

fix(rust/sedona-raster-gdal): read a file nodata the band type cannot hold as none - #1357

Merged
james-willis merged 1 commit into
apache:mainfrom
james-willis:james/gdal-nodata-representable
Sep 30, 2026
Merged

james-willis merged 1 commit into
apache:mainfrom
james-willis:james/gdal-nodata-representable

Conversation

@james-willis

Copy link
Copy Markdown
Contributor

The GDAL loader packed a file's nodata into the band's data type with saturating as casts. GDAL reports nodata as an f64 (for GeoTIFF, the ASCII TIFFTAG_GDAL_NODATA tag), so a file can declare a value its band type cannot hold, and the cast turned that value into a real pixel value. Every pixel equal to the result then read as nodata. Before this change, on a 2x2 UInt8 GeoTIFF with pixels 0, 255, 1, 7:

file nodata SedonaDB RS_BandNoDataValue SedonaDB reads as NULL Sedona Spark RS_BandNoDataValue Spark reads as NULL
-9999 0 the 0 pixel -9999 nothing
256 255 the 255 pixel 256 nothing
NaN 0 the 0 pixel NULL nothing
0.5 0 the 0 pixel 0.5 nothing

Both loaders (out-db RS_FromPath, in-db RS_FromGDALRaster) go through band_nodata_to_bytes, so every consumer of band nodata was affected: RS_Value/RS_Values, RS_BandNoDataValue, RS_MinConvexHull, and the pending RS_BandIsNoData (#1350) and RS_SummaryStats (#1352).

Semantics

A band now has no nodata when the file's value is not exactly representable in the band type. No pixel can equal such a value, and that is how the other readers treat it:

  • Sedona Spark keeps the declared double, so it matches no pixel.
  • GDAL's mask band is all_valid for an out-of-range or NaN nodata (checked with GDAL 3.12.4), and rasterio's nodatavals reports None for one.

Raising an error instead would refuse files that GDAL, rasterio and Spark all open, and a user cannot fix a file's metadata from SQL. That is the difference from sedona_raster::traits::nodata_f64_to_bytes, which errors because its input is a query argument (RS_SetBandNoDataValue, RS_Clip, RS_Tile). The loader now uses that same exact check and treats its error as "no nodata". The saturating converter in gdal_common.rs, and its re-export, are removed.

Two known differences remain:

  • In-range fractions. GDAL's mask band truncates a fractional nodata on an integer band (0.5 masks 0, -1.5 masks -1). Here it matches no pixel, as in Spark, so one exactness rule covers every case.
  • RS_BandNoDataValue. It returns NULL where Spark reports the declared -9999, 256 or 0.5, because SedonaDB stores a band's nodata in the band type. For NaN on an integer band, both engines now return NULL.

Float bands are unchanged: Float32 rounds to the nearest f32, as GDAL does, and NaN stays NaN. UInt64/Int64 go through GDAL's exact 64-bit getters and are unchanged too.

Tests

  • Rust: test_band_nodata_to_bytes_drops_unrepresentable_nodata runs over MEM bands. Values below or above the range, NaN, and fractions are dropped. Range bounds and a Float32 value are kept.
  • python/sedonadb/tests/functions/test_rs_value.py: test_unrepresentable_file_nodata_is_dropped runs through both loaders. RS_BandNoDataValue is NULL, and every pixel samples verbatim, including one planted at the value the old cast produced. It is anchored to the data, which rasterio's comparator also returns. The file's fixtures switch from a raw geotransform to the equivalent bbox.
  • write_geotiff now sets nodata through rasterio's band setter. Unlike rasterio.open(nodata=), the setter accepts a value outside the dtype's range, and it writes byte-identical files for every other value. test_raster_testing.py checks that the value lands in the file by reading it back through a GDAL VRT copy, because rasterio hides an out-of-range nodata.
  • spark-parity test_rs_value.py: a pixel-level case over -9999, 256, NaN and 0.5 now passes on both engines (the planted pixel reads verbatim).
  • spark-parity test_rs_bandnodatavalue.py: NaN on uint8 passes, anchored to NULL. The fractional xfail becomes one parametrized xfail over -9999, 256 and 0.5, with the reason updated to the NULL-vs-verbatim divergence.

Once this lands, #1352's test_rs_summarystats_fractional_nodata_on_int_band xfail will XPASS and can be flipped.

… hold as none

GDAL reports a band's nodata as an f64 (for GeoTIFF, the ASCII
TIFFTAG_GDAL_NODATA tag), so a file can declare -9999 or NaN on a UInt8
band, or 0.5 on an Int32 band. The loader packed it into the band type
with saturating `as` casts: -9999 and NaN became 0, 256 became 255, 0.5
became 0, and every real pixel equal to the result read as nodata
(RS_Value returned NULL for real 0 pixels).

No pixel can equal such a value, so the band now reads as having no
nodata. The GDAL-side conversion is replaced by the exact check in
sedona_raster::traits::nodata_f64_to_bytes, with its error mapped to
"no nodata" rather than failing the read: the file is valid, and GDAL,
rasterio and Sedona Spark all open it. Sedona Spark matches no pixel
either, and GDAL's own mask band ignores an out-of-range nodata the
same way.

write_geotiff now sets nodata through rasterio's band setter, which
unlike the open() argument accepts values outside the dtype's range
(files are byte-identical otherwise), so tests can write these files.
@james-willis
james-willis marked this pull request as ready for review September 25, 2026 20:08

@paleolimbot paleolimbot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@james-willis
james-willis merged commit 7517056 into apache:main Sep 30, 2026
17 checks passed
james-willis added a commit to james-willis/sedona-db that referenced this pull request Sep 30, 2026
…fixed

Loading a raster whose file nodata is not representable in the band dtype now
drops the nodata rather than packing it (apache#1357), so SedonaDB no
longer turns 0.5 into 0 and no longer excludes real 0 pixels. Both engines
agree, and strict xfail surfaced the marker as stale.
@james-willis
james-willis deleted the james/gdal-nodata-representable branch October 1, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants