Add a worked example of three extension libraries in one distributed query - #1721

Draft

timsaucer wants to merge 24 commits into

feat/plan-partitioning-and-errorsfrom

feat/distributed-extensions-example

Commits on Sep 10, 2026

Add dfx_storage: a codec that writes durable metadata

timsaucer
and
claude
committed
6b18896
Add dfx_udfs: the library that cannot be installed as a bundle

timsaucer
and
claude
committed
532995c
Add dfx_engine: a toy distributed engine in both halves

timsaucer
and
claude
committed
dba062c
Add the queries and the cross-library integration tests

timsaucer
and
claude
committed
f8ca9b0
Wire the distributed example into CI, and say which example is which

timsaucer
and
claude
committed
9339ac8
Document what a worker has to reproduce, and three findings that cost time

timsaucer
and
claude
committed
467fef1
Build the example libraries on the host, and prime their deps once

timsaucer
and
claude
committed
05b40fb
Number the stages, and ship all of them

timsaucer
and
claude
committed
016ad9d
Read the third codec id instead of copying it

timsaucer
and
claude
committed
3a61caf
Make the example's run instructions work

timsaucer
and
claude
committed
e3ae58e
Cover build_session's own consistency check

timsaucer
and
claude
committed
6e3dc96
Point local_session.rs at a section that exists

timsaucer
and
claude
committed
be191ac
Say why table_options_mut panics on purpose

timsaucer
and
claude
committed
4662e18
Link the two sections this PR added

timsaucer
and
claude
committed
60bb0f8
Decode a provider with the schema from the plan

timsaucer
and
claude
committed
1774343
Refuse a malformed projection instead of dropping it

timsaucer
and
claude
committed
628c451
Raise Execution, not Internal, for a bad payload

timsaucer
and
claude
committed
22912ac
Stop converting a fallback this planner never calls

timsaucer
and
claude
committed
9f9f688
Clear the small stuff off the distributed example

timsaucer
and
claude
committed
4a8cb2c

Commits on Sep 11, 2026

Refuse a shuffle directory that already holds stage output

timsaucer
and
claude
committed
b0030c7
Check a shuffle file's schema instead of adopting it

timsaucer
and
claude
committed
089a856
Size a scalar function's output from the batch, not its arguments

timsaucer
and
claude
committed
d76822f
Say what the example-test gate actually does

timsaucer
and
claude
committed
936cb1b
Keep the driver's scratch out of the results directory

timsaucer
and
claude
committed
857a9ff

Navigation Menu

Uh oh!

Add a worked example of three extension libraries in one distributed query - #1721

Add a worked example of three extension libraries in one distributed query#1721
timsaucer wants to merge 24 commits into
feat/plan-partitioning-and-errorsfrom
feat/distributed-extensions-example

Commits on Sep 10, 2026

Add dfx_storage: a codec that writes durable metadata

Add dfx_udfs: the library that cannot be installed as a bundle

Add dfx_engine: a toy distributed engine in both halves

Add the queries and the cross-library integration tests

Wire the distributed example into CI, and say which example is which

Document what a worker has to reproduce, and three findings that cost time

Build the example libraries on the host, and prime their deps once

Number the stages, and ship all of them

Read the third codec id instead of copying it

Make the example's run instructions work

Cover build_session's own consistency check

Point local_session.rs at a section that exists

Say why table_options_mut panics on purpose

Link the two sections this PR added

Decode a provider with the schema from the plan

Refuse a malformed projection instead of dropping it

Raise Execution, not Internal, for a bad payload

Stop converting a fallback this planner never calls

Clear the small stuff off the distributed example

Commits on Sep 11, 2026

Refuse a shuffle directory that already holds stage output

Check a shuffle file's schema instead of adopting it

Size a scalar function's output from the batch, not its arguments

Say what the example-test gate actually does

Keep the driver's scratch out of the results directory