Repository navigation
write_elem and read_elem: an EHRData as part of a larger HDF5 file or Zarr store - #360
Merged
Merged
Conversation
… stores write_elem writes an EHRData into a group in the same layout as write_h5ed and write_zarr; read_elem reads it back. The file-level functions now use the same code, and the on-disk format is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As anndata does, so a package bundling an EHRData can keep its own metadata in the group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #357.
Packages that bundle an
EHRDatawith other objects in one file, such as a DonorData in a.dd.h5/.dd.zarr, couldn't write it into a group or read it from there.write_h5ed,read_h5edandwrite_zarronly took paths. The workarounds were temporary files or a copy of ehrdata's private on-disk layout.New
parent[key]gets exactly whatwrite_h5ed(HDF5) orwrite_zarr(Zarr) write to a file. That includes 3D arrays in the reservedobsmkeys, sparse 3D COO,tem, the encoding attributes, the column casting andconvert_strings_to_categoricals.write_h5ed/read_h5ed/write_zarr/read_zarrnow call the same code. Backedread_h5edis unchanged.compression/compression_optsapply only to HDF5, andchunksonly to Zarr. Passing an option for the other format raises.chunks="ehrdata_auto"needs v3 and raises before writing anything.anndata.io.write_elem./; pass the nested group asparentinstead.write_h5ed/write_zarr.parent[key]intact.read_elemon a group that holds no EHRData names the group in its error.read_elemignores group children other than the AnnData slots andtem, as anndata does. A package bundling an EHRData can keep its own metadata in the group.On-disk format
Unchanged. Files and stores written with
mainand with this branch were compared member by member: tree, attributes, dtypes, shapes, compression, chunks/shards and values. The comparison covered dense and COO 3D, CSR, object columns and layers, gzip and"ehrdata_auto", with conversion on and off, and found no differences. Each version reads the other's files to the sameEHRData.One detail:
ad.io.write_elemwrites a nullrawthatwrite_h5adnever did, so.h5edgroups drop it after writing.Also
chunkstype hint was fixed:Literal["auto", "ehrdata_auto"].write_zarrdocstring example was fixed.read_elemandwrite_elem.Tests
tests/io/test_elem.pyruns on an HDF5 file, a Zarr v3 group and a Zarr v2 group, and reuses the existing fixtures. It covers:temand the column casting;convert_strings_to_categoricals;write_h5ed/write_zarrproduce;🤖 Generated with Claude Code