[alexanderbianchi]

Which issue does this PR close?

Related to #3276 and #3277. No issue is closed by this draft.

Depends on #3587. This branch currently includes that prerequisite commit. The data-file change is the second commit; review the incremental diff here. Rebase onto main after #3587 merges.

What changes are included in this PR?

Data-file footer and page-index decoding currently run on I/O workers, while page decoding runs on whichever runtime polls the scan. Move the complete file-open/planning phase and Parquet stream consumption onto the configured CPU runtime. Storage opens and byte-range reads continue to use the I/O adapter from #3587.

Keep opening/planning in spawned, abort-on-drop tasks so buffered opens and bloom-filter fetches continue progressing when the consumer polls another file. Run bloom-filter evaluation directly inside that CPU task.

Deliver decoded batches through a bounded channel per file. Reserve capacity before polling Parquet, allowing at most one output batch ahead of the consumer. Retain an abort-on-drop producer handle with the returned stream so dropping a scan cancels outstanding work. Await the producer at EOF to propagate decoding failures, panics, and runtime shutdown instead of silently truncating results.

Existing file-concurrency controls are retained. The channel bounds output batches, not Parquet's internal row-group buffers. This adds CPU task/channel overhead; no throughput benchmark has been run.

Are these changes tested?

• cargo test -p iceberg --lib arrow::: 291 passed.
• cargo test -p iceberg --lib scan::: 104 passed.
• New tests cover CPU stream polling with an I/O-runtime consumer, bounded decoding ahead of the consumer, cancellation while waiting for capacity or decoding, propagation of decoding errors and panics, and a real Parquet scan with separate runtimes that asserts storage opens and reads execute on I/O workers.
• Existing open-progress and cancellation regressions from fix(arrow): spawn data file opens on the IO runtime #3277 pass.
• cargo clippy -p iceberg --lib --tests -- -D warnings: passed.
• cargo fmt --all -- --check: passed.

AI Disclosure

Implementation, tests, and this description were prepared with OpenAI Codex assistance.