GH-51138: [Python] Fix ccache efficiency by pitrou · Pull Request #51139 · apache/arrow

pitrou · 2026-09-02T13:34:05Z

Rationale for this change

Our current ccache support for PyArrow currently has two issues:

The custom configuration is set using environment variables, but those are not passed to the build processes according to the CMake doc (I have verified this by enabling ccache logging to see the configuration values for each build command)
Cython-generated C++ sources are written in the temporary build directory (because of build isolation), and therefore their path changes everytime. Consequently, ccache would fail reusing previous compilation results.

What changes are included in this PR?

Pass custom ccache configuration using command-line arguments, not environment variables.
Set the ccache configuration variable base_dir to the temporary build directory so that ccache strips away the build directory and hits previously cached results obtained from a different build directory.

Are these changes tested?

Yes, locally I confirmed that ccache now efficiently reuses compilation outputs for Cython-generated C++ sources.

Before:

real    0m50,313s
user    4m22,991s
sys     0m24,805s

After:

real    0m18,698s
user    1m14,727s
sys     0m7,518s

Are there any user-facing changes?

No.

GitHub Issue: [Python] ccache not efficient #51138

github-actions · 2026-09-02T13:35:47Z

⚠️ GitHub issue #51138 has been automatically assigned in GitHub to PR creator.

pitrou · 2026-09-02T14:51:36Z

@github-actions crossbow submit -g python

github-actions · 2026-09-02T14:54:49Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-290df36268

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.3
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

rok · 2026-09-02T15:22:48Z

@github-actions crossbow submit test-ubuntu-22.04-cpp -p image=ubuntu-python

github-actions · 2026-09-02T15:25:19Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-ba899c1522

Task	Status
test-ubuntu-22.04-cpp

pitrou · 2026-09-02T16:22:28Z

@github-actions crossbow submit test-ubuntu-22.04-cpp -p image=ubuntu-python

What's this @rok ?

rok · 2026-09-02T17:30:57Z

+  # 2. Set ccache base_dir to the build output directory as it is typically
+  #    a temporary directory, and would otherwise fail caching because of
+  #    using different paths everytime.
+  set(ccache_command ${CCACHE_FOUND} keep_comments_cpp=true base_dir=${CMAKE_BINARY_DIR})


As per my 🤖 ccache pre-4.8 does not accept compiler options (keep_comments_cpp=true in this case). And testing on an older ccache I get:

> ccache keep_comments_cpp=true base_dir=/tmp gcc --version ccache: error: Could not find compiler "keep_comments_cpp=true" in PATH

So users with old ccache building pyarrow would have this crash.

Hmm, are there any Python builds which use an old ccache?

I think no, but users might. I think this is obscure enough to ignore.

If there was an easy way to query the ccache version we could work around this to ensure a better UX. But I'm not sure how to do that. @kou ?

(for the record, ccache 4.8 was released in March 2023)

@kou How about cmake -E env as suggested by @rok above?

$ cmake -E env CCACHE_BASEDIR=/top -- ccache -p | \grep base_dir (environment) base_dir = /top

It can avoid the error but meaningless ccache command is still used (ccache storage is consumed). If it's acceptable, we can use the cmake -E env approach.

Why "meaningless"?

Because:

Cython-generated C++ sources are written in the temporary build directory (because of build isolation), and therefore their path changes everytime. Consequently, ccache would fail reusing previous compilation results.

We're setting CCACHE_BASEDIR precisely to avoid that.

pitrou · 2026-09-02T17:32:30Z

@github-actions crossbow submit wheelcp311*

github-actions · 2026-09-02T17:35:17Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-131e0bf55a

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

pitrou · 2026-09-03T08:33:57Z

@github-actions crossbow submit -g python

pitrou · 2026-09-03T08:34:04Z

@github-actions crossbow submit wheelcp311*

github-actions · 2026-09-03T08:37:27Z

Revision: a1df242

Submitted crossbow builds: ursacomputing/crossbow @ actions-db9b456f6a

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

github-actions · 2026-09-03T08:38:09Z

Revision: a1df242

Submitted crossbow builds: ursacomputing/crossbow @ actions-695eab7643

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.3
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

pitrou · 2026-09-03T09:15:48Z

I'm going to merge as CI failures are unrelated.

github-actions Bot added the awaiting review Awaiting review label Sep 2, 2026

github-actions Bot added the Component: Python label Sep 2, 2026

pitrou force-pushed the gh51138-python-ccache branch from 9a5a1b7 to a66cc75 Compare September 2, 2026 13:46

pitrou mentioned this pull request Sep 2, 2026

[Python] Adopt new features and bugfixes from Cython 3.1 #47467

Open

pitrou marked this pull request as ready for review September 2, 2026 14:51

pitrou requested review from AlenkaF, assignUser, jonkeane, kou, raulcd and rok as code owners September 2, 2026 14:51

rok reviewed Sep 2, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Sep 2, 2026

rok approved these changes Sep 2, 2026

View reviewed changes

github-actions Bot added awaiting merge Awaiting merge awaiting changes Awaiting changes and removed awaiting changes Awaiting changes awaiting merge Awaiting merge labels Sep 2, 2026

kou added the CI: Extra: CUDA Run extra CUDA CI label Sep 3, 2026

pitrou added 2 commits September 3, 2026 10:28

apacheGH-51138: [Python] Fix ccache efficiency

298337a

Use cmake -E env to set ccache configuration options

a1df242

pitrou force-pushed the gh51138-python-ccache branch from a66cc75 to a1df242 Compare September 3, 2026 08:33

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 3, 2026

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Sep 3, 2026

pitrou merged commit 503cee9 into apache:main Sep 3, 2026
42 of 44 checks passed

pitrou removed the awaiting changes Awaiting changes label Sep 3, 2026

pitrou deleted the gh51138-python-ccache branch September 3, 2026 09:16

pitrou mentioned this pull request Sep 3, 2026

[Python] ccache not efficient #51138

Closed

Uh oh!

Conversation

pitrou commented Sep 2, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

rok commented Sep 2, 2026

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pitrou commented Sep 2, 2026 •

edited by github-actions Bot

Loading