Repository navigation
EPIC: Add all SessionContext and DataFrame methods to Python API #24
Description
The Python bindings currently only expose a subset of functionality, and we want to expose as much as possible.
Here is a list of all available rust methods. Note that there may be reasons why we don't want to expose some of these.
SessionContext
- new
- with_config
- register_csv
- register_parquet
- sql
- catalog
- table_exist
- table
- tables
- register_table
- register_udf
- register_udaf
- Add Python bindings for
SessionContextmethods read_csv, read_json, read_parquet, read_avro #28 - register_variable
- read_table
- Add
read_jsontoSessionContext#56 - [SessionContext] - Add read_csv/read_parquet/read_avro functions to SessionContext #57
- [DataFrame] - Add write_csv/write_parquet/write_json to DataFrame #58
- register_listing_table
- register_json
- register_avro
- register_catalog
- deregister_table
- session_id
- state
DataFrame
- new
- select
- select_columns
- with_column
- with_column_renamed
- explain
- collect
- show / show_limit
- schema
- filter
- aggregate
- sort
- join
- limit
- Add Python bindings for DataFrame methods
intersectandexcept#27 - Add Python bindings for DataFrame methods
unionandunion_distinct#25 - Add Python binding for
DataFrame::distinct#29 - Add Python binding for
DataFrame::repartition#30 - Add Python bindings for DataFrame methods write_csv, write_parquet, and write_json #26
- Implement DataFrame execute_stream and execute_stream_partitioned #12
- collect_partitioned
- to_logical_plan
- create_physical_plan
- registry
Activity
[-]EPIC: Add all DataFrame methods to Python API[/-][+]EPIC: Add all `SessionContext` and `DataFrame` methods to Python API[/+]francis-du commentedon Nov 2, 2022
@andygrove Hi Andy. Could you update this task list? If you have a time.
nkarpov commentedon Apr 12, 2023
Noticing that
join_on(supporting arbitrary expressions instead of keys) is not present in the list above.I can't find a way to use the existing binding to join on an arbitrary expression
Should the Rust join_on have its own wrapper too? or should we have the existing
joinwrapper choose appropriately?ami-navon commentedon Jun 5, 2023
+1 register_json( )
timsaucer commentedon Apr 9, 2026
Closed by #1475 and #1472