Keep the floating dtype of the input in the host round trips

[mwcraig]

Fixes #1023

Commit	What it does
b078791	Return host results in the floating dtype of the input (_from_numpy rule, wcs_project)
69f7eda	Give subtract_overscan's fitted overscan the plain path's dtype (like=oscan)
4dd2768	Keep the pixels astroscrappy does not flag exactly (reverted by ff0fb2f)
5e0245c	Record the host round trips' dtype rule in CHANGES and the docs
bbeb7e9	Keep the dtype of files that combine reads next to images in memory
ff0fb2f	Take cosmicray_lacosmic's whole result from astroscrappy again (reverts 4dd2768 and its two tests)
d4fd577	Trim the dtype notes in the subtract_overscan and wcs_project docstrings
c62c932	Shorten the #1023 CHANGES entry
b52bb17	Let the ccd_data test helper make an image of a given dtype
ed91943	Keep the uncertainty's dtype when cosmicray_lacosmic applies the gain
185b745	Make the lacosmic gain without a branch for integer uncertainties
c252b05	Point the strict integer xfails at the new issue #1034
4423146	Pass a float32 invar to astroscrappy in cosmicray_lacosmic

The rule

The three functions that do their work in a NumPy-only library (wcs_project, subtract_overscan(model=), cosmicray_lacosmic) used to return whatever dtype that library produced. Now:

• Floating input keeps its dtype. float32 stays float32 and float64 stays float64. For cosmicray_lacosmic, float64 output holds float32-precision values, since astroscrappy works in float32.
• Integer input keeps today's floating result. That is float64 from wcs_project, and from subtract_overscan the same dtype with or without a model.
• cosmicray_lacosmic with integer input stays float32, whatever the gain.
• With gain_apply=True, the lacosmic uncertainty keeps its own dtype.

Output dtype, before → after (NumPy)

CCDData of about 1000 ADU, 40×40. For the uncertainty row, the uncertainty is float32, except float64 for float64 data.

call	float32	float64	uint16	int32
subtract_overscan mean / median (unchanged)	f32	f64	f64	f64
subtract_overscan(model=)	f64 → f32	f64	f64	f64
wcs_project	f64 → f32	f64	f64	f64
cosmicray_lacosmic data (CCDData with gain 1 or 2; bare array)	f32	f32 → f64	f32	f32
lacosmic uncertainty, gain_apply=True	f64 → f32	f64	f64 → f32	f64 → f32

Other backends

• dask: identical to NumPy in every cell, before and after.
• array-api-strict (device1):
  • Identical to NumPy for float32 and float64.
  • For uint16 and int32, subtract_overscan raises TypeError on every path, before and after (pre-existing bug 1).
  • wcs_project and lacosmic data dtypes are as on NumPy.
  • lacosmic with gain_apply=True on integer data with an uncertainty raises TypeError, before and after (pre-existing bug 2).
• JAX (x64):
  • Same as NumPy, except integer overscan: the mean and median paths give f32 for uint16 and int32, before and after. model= went f64 → f32, so it now matches them (question 2).
  • The lacosmic uncertainty already kept its own dtype.
• Byte order: big-endian FITS input (>f4, >f8) comes back native, in the same float dtype, from all three functions.

Implementation notes

• _from_numpy: when arr and like are both real floating, the result gets like's dtype. Otherwise it keeps arr's dtype, which covers boolean masks, results for integer input, and like=None. Documented in its Returns and Notes and pinned by a 6-case test.
• subtract_overscan converts the fitted model with like=oscan, the reduced overscan, rather than like=ccd.data. The reduced overscan is always floating, so the fit comes back in the dtype the mean and median paths subtract, including for integer data, where that dtype depends on the array library.
• cosmicray_lacosmic gain: gain_value is made in xp.result_type(<uncertainty dtype>, float32): a floating uncertainty's own dtype, and a floating gain for an integer uncertainty (185b745). Using the cleaned data's dtype everywhere, as first proposed in the thread, would still have turned a float32 uncertainty with float64 data into float64, and rounded the gain to float32 for a float64 uncertainty.
• combine (bbeb7e9): after rebasing onto Drop the unreleased public xp keyword and raise on mixed array namespaces or devices #1028, _ccddata_from_numpy went through _from_numpy, and the dtype rule cast a file read by combine to the floating dtype of the first image in memory. _ccddata_from_numpy now converts with xp.asarray on like's device and keeps each array's dtype.

Tests added or changed

• pytest_fixtures.py: ccd_data takes dtype=.
• test_hostcopy.py
  • test_from_numpy_restores_namespace_device_and_dtype: 6 cases (float both ways, bool mask, integer like); it used to pin the old pass-through.
  • test_wcs_project_keeps_floating_dtype
  • test_subtract_overscan_model_keeps_dtype_of_plain_path, with a non-strict backend_xfail for uint16 and int32 on array-api-strict (array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034).
• test_cosmicray.py
  • test_cosmicray_lacosmic_keeps_floating_dtype (CCDData and bare array)
  • test_cosmicray_lacosmic_integer_input_gives_float32 (gain 1 and 2)
  • test_cosmicray_lacosmic_gain_apply_keeps_uncertainty_dtype (float32/float64/uint16 data, float32 uncertainty; strict uint16 is a non-strict xfail linking array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034, because the integer data itself is multiplied by a floating scalar)
  • test_cosmicray_lacosmic_float64_invar
• test_combiner.py: test_combine_mixed_list_keeps_float64_file_precision

Questions (all resolved)

1. Where the cast lives. Resolved: keep the rule in _from_numpy. It matters at three call sites (wcs_project data, the overscan fit, lacosmic data), and all three want exactly this. _ccddata_from_numpy bypasses it.
2. like=oscan in subtract_overscan. Resolved: accepted. Side effect: JAX integer input with a model now gives float32 instead of float64, matching JAX's mean and median paths (unreleased, so no CHANGES line).
3. Editing the unreleased Make the host round trip explicit in wcs_project, subtract_overscan and cosmicray_lacosmic #1011 CHANGES entry. Resolved: keep the edit.
4. lacosmic uncertainty dtype. Resolved: fixed here (ed91943, 185b745).
5. Integer overscan on array-api-strict. Resolved: opened array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034 (it also covers the strict lacosmic gain_apply case), both strict xfails now link it (c252b05), and a note is posted on Tracking: remaining work for the array-API migration #971.
6. A float64 invar makes astroscrappy raise. Resolved: fixed here (4423146). astroscrappy converts inbkg to float32 itself but not invar, and its cleaning routines accept only float32, so every cleantype raised "Item size of buffer (8 bytes) does not match size of 'float32_t'". The host copy of invar is now cast to float32. A float64 inbkg already worked.

Pre-existing bugs found (not caused by this PR)

1. array-api-strict, integer subtract_overscan fails on every device, with or without model=. array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034.
2. array-api-strict, cosmicray_lacosmic on integer data with an uncertainty and gain_apply=True raises "uint16 and float32 cannot be type promoted together" (was "... float64 ..."): the integer data itself is multiplied by a floating scalar. array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034.
3. The lacosmic uncertainty dtype: fixed here.
4. A float64 invar is rejected inside astroscrappy: fixed here (4423146).
5. JAX without x64 (Decide whether jax without JAX_ENABLE_X64 is a supported test configuration #1020). The new dtype tests assume JAX_ENABLE_X64=True, like the rest of the suite.
6. numpy200 env did not test numpy 2.0: fixed by the SPEC 0 PR Raise minimum versions to follow SPEC 0 (Python 3.12, numpy 2.2, astropy 7.0, scipy 1.15) #1031.
7. astropy's CCDData.__init__ calls np.asarray on the mask, so a CCDData with an array-api-strict mask fails before ccdproc is called. That is on astropy's side.

Local results

At 4423146, full suites:

env	result
py312-test-alldeps	1403 passed, 67 skipped
py313-strict	1417 passed, 48 skipped, 5 xfailed
py313-jax	1410 passed, 60 skipped
py312-alldeps-dask	1412 passed, 58 skipped
py312-alldeps-dask-enforce	1412 passed, 58 skipped, "OK: no library escapes outside the baseline."
py312-alldeps-dask-bottleneck	1414 passed, 56 skipped
py312-test-alldeps-numpy220-bottleneck	1409 passed, 61 skipped
codestyle	passed
build_docs	passed

Review process

The branch was written, then went through two rounds of adversarial review by fresh subagents with a revision after each; those fixups are squashed into the first four commits. After a rebase, the one semantic interaction it exposed got its own commit (bbeb7e9). The maintainer's review round was then applied as 5 new commits, followed by an adversarial review. The answers to questions 2, 5 and 6 added 185b745, c252b05 and 4423146.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RgzdGvWt8d9CYfT2ayUpxz

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.45%. Comparing base (bc9f459) to head (4423146).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1032   +/-   ##
=======================================
  Coverage   98.44%   98.45%           
=======================================
  Files          11       11           
  Lines        2258     2267    +9     
=======================================
+ Hits         2223     2232    +9     
  Misses         35       35           

Flag	Coverage Δ
dask	97.70% <100.00%> (+<0.01%)	⬆️
jax	97.65% <100.00%> (+<0.01%)	⬆️
numpy	97.48% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation is internally consistent and comprehensively covers the affected dtype, precision, backend, and lazy-evaluation behavior.

Review effort: Balanced
Findings: None

What changed in this PR

Preserves floating-point dtypes across NumPy-only host round trips without affecting integer-result behavior.

Changes:

• Applies dtype-aware host-result conversion.
• Preserves unflagged L.A.Cosmic pixels exactly.
• Keeps FITS precision during mixed-input combination and adds regression coverage.

File	Description
ccdproc/​core.py	Implements dtype preservation and exact pixel merging.
ccdproc/​tests/​test_hostcopy.py	Tests host conversion and projection/overscan dtypes.
ccdproc/​tests/​test_cosmicray.py	Tests L.A.Cosmic dtype, precision, and lazy evaluation.
ccdproc/​tests/​test_combiner.py	Tests mixed-source precision preservation.
docs/​array_api.rst	Documents host-round-trip dtype behavior.
CHANGES.rst	Records user-visible behavior changes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mwcraig]

Fix 4 here if the fix is one line

[mwcraig]

Fix 4 here if the fix is one line

It is one line. In the gain_apply block of cosmicray_lacosmic, add dtype=cleanarr.dtype to the xp.asarray that makes gain_value:

gain_value = xp.asarray(
    float(gain.value),
    dtype=cleanarr.dtype,
    device=array_api_compat.device(_ccd.data),
)

cleanarr is always floating: the input's dtype if that is floating, otherwise float32. I checked it on a scratch copy of bbeb7e9, with a 40×40 CCDData at 1000 ADU, gain 2 e/ADU and gain_apply=True. Here is the uncertainty dtype for each data dtype, before → after:

data	uncertainty in	NumPy	dask	array-api-strict (device1)	JAX
float32	float32	f64 → f32	f64 → f32	f64 → f32	f32
float64	float64	f64	f64	f64	f64
uint16 / int32	float32	f64 → f32	f64 → f32	TypeError, still	f32
any	float64	f64	f64	f64 (TypeError for integer data)	f64

The data dtype doesn't change in any cell. The uncertainty now keeps its own dtype on every backend, which is what JAX already did.

It does not fix array-api-strict with integer data (pre-existing bug 2). The error changes from "uint16 and float64 cannot be type promoted" to "uint16 and float32 cannot be type promoted", because that case multiplies the integer data itself by a floating scalar. That's the #1013 family and needs more than one line, so I'd leave it out of this PR.

I'll add the line, plus a parametrized test that pins the uncertainty dtype for float32 and uint16 data (strict uint16 as an xfail pointing at the bug), when I make the other changes from this review. I haven't changed anything yet.

— Written by Claude at @mwcraig's direction.

[mwcraig]

Fix 4 here if the fix is one line

Done in ed91943, but not quite with the line I proposed above. Review of the new commits found that dtype=cleanarr.dtype still misbehaves when the data and the uncertainty have different dtypes:

• float64 data with a float32 uncertainty: the uncertainty still came back float64.
• float32 or integer data with a float64 uncertainty: the gain was rounded to float32 first, so a gain of 1.1 turned an uncertainty of 10 into 11.000000173 instead of 11.0.

The gain now takes the uncertainty's own dtype, so multiplying by it can't change that dtype or lose precision. If the uncertainty isn't floating, it falls back to the cleaned data's dtype. That makes it four lines instead of one. Only the uncertainty comes from that multiplication, so the data dtype is unaffected.

test_cosmicray_lacosmic_gain_apply_keeps_uncertainty_dtype pins float32, float64 and uint16 data with a float32 uncertainty. As before, strict uint16 is a non-strict xfail (the integer data itself is multiplied by a floating scalar). Its reason links #1013, which is the nearest issue but is really about subtract_dark. If you'd rather have a dedicated issue, one issue for "array-api-strict: integer data with floating scalars" could cover this xfail and the integer-overscan one (question 5).

I've updated the PR description to match the branch as it is now (no longer stacked, merge removed, question 4 resolved).

— Written by Claude at @mwcraig's direction.

[mwcraig]

Following up on questions 2, 5 and 6 from the description, pushed as 185b745..4423146:

• 2: like=oscan stays as it is. No change.
• 5: opened array-api-strict: subtract_overscan and cosmicray_lacosmic(gain_apply=True) fail on integer data #1034 for array-api-strict failing on integer data, in both subtract_overscan and cosmicray_lacosmic(gain_apply=True). Both strict xfails now link it (c252b05), and a note is on Tracking: remaining work for the array-API migration #971.
• 6: fixed here (4423146). astroscrappy converts inbkg to float32 itself but not invar, and its cleaning routines accept only float32. So a float64 invar made every cleantype raise "Item size of buffer (8 bytes) does not match size of 'float32_t'". The host copy of invar is now cast to float32. test_cosmicray_lacosmic_float64_invar fails without the fix, and there's a CHANGES bug-fix line.

Also, 185b745 replaces the untested fallback branch that codecov flagged with xp.result_type(<uncertainty dtype>, float32).

The description is updated. Question 1 (where the cast lives) is still open.

— Written by Claude at @mwcraig's direction.