[parshimers]

What changes were proposed in this PR?

Previously, a variety of things caused Texera images built
from the same source to not have the same binary output from
invocation to invocation. Having this kind of binary
build reproducibility is a requirement when performing
Apache releases not on developer-owned hardware, because the
release has to be verified on developer owned hardware by
rebuilding it and getting the exact same artifact.

This change allows all release artifacts to be binary
reproducible with effectively one piece of information,
though in two different formats. These build arguments
contain the date: SOURCE_DATE_EPOCH and PACKAGE_SNAPSHOT.
The first sets a uniform timestamp wherever a timestamp
is present, and the second sets the snapshot of the apt
repositories used in image builds.

One other piece of information is implicit, but important
in practice. Binary reproducibility is inherently dependent
on the versions of tools used to produce said binaries.
Most of them are pinned within the container build files, but
the container builder itself is another piece of necessary
information (e.g. buildah vs. buildx) for reproducing
a build from its source and time.

Any related issues, documentation, discussions?

Closes #8809. Discussion: #8807.

How was this PR tested?

bin/README.md documents what is pinned and how to rebuild an image; the vote email gives voters the command that rebuilds both tarballs.

Meng and I went back and forth, building all of the release images on each of our machines (an ARM Mac, and a x86 Linux machine) and compared the hashes each of us got. We eventually got all of them to line up.

Was this PR authored or co-authored using generative AI tooling?

Yes, Claude Opus 5.5

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Ma77Ball, @bobbai00, @Yicong-Huang
  You can notify them by mentioning @Ma77Ball, @bobbai00, @Yicong-Huang in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.04%. Comparing base (92e86d4) to head (853ef7c).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8812      +/-   ##
============================================
+ Coverage     91.98%   92.04%   +0.06%     
+ Complexity     5083     5038      -45     
============================================
  Files          1285     1286       +1     
  Lines         56094    55920     -174     
  Branches       7137     7137              
============================================
- Hits          51599    51473     -126     
+ Misses         2742     2693      -49     
- Partials       1753     1754       +1     

Flag	Coverage Δ
access-control-service	77.44% <ø> (ø)
agent-service	99.16% <ø> (ø)
amber	86.55% <ø> (+0.11%)	⬆️
computing-unit-managing-service	64.02% <ø> (ø)
config-service	87.61% <ø> (ø)
file-service	82.29% <ø> (ø)
frontend	96.47% <100.00%> (+<0.01%)	⬆️
notebook-migration-service	83.77% <ø> (ø)
pyamber	98.54% <ø> (ø)
workflow-compiling-service	74.25% <ø> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 11 worse · ⚪ 4 noise (<±5%) · 0 without baseline

Compared against main 92e86d4 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	326	0.199	30,704/39,343/39,343 us	🔴 +48.9% / 🔴 +173.4%
🔴	bs=100 sw=10 sl=64	812	0.495	118,902/160,971/160,971 us	🔴 +14.5% / 🔴 +63.9%
🔴	bs=1000 sw=10 sl=64	958	0.585	1,026,719/1,204,331/1,204,331 us	🔴 +10.6% / 🔴 +28.0%

Baseline details

Latest main 92e86d4 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	326 tuples/sec	422 tuples/sec	879.43 tuples/sec	-22.7%	-62.9%
bs=10 sw=10 sl=64	MB/s	0.199 MB/s	0.258 MB/s	0.537 MB/s	-22.9%	-62.9%
bs=10 sw=10 sl=64	p50	30,704 us	20,622 us	11,622 us	+48.9%	+164.2%
bs=10 sw=10 sl=64	p95	39,343 us	34,232 us	14,390 us	+14.9%	+173.4%
bs=10 sw=10 sl=64	p99	39,343 us	34,232 us	17,721 us	+14.9%	+122.0%
bs=100 sw=10 sl=64	throughput	812 tuples/sec	870 tuples/sec	1,166 tuples/sec	-6.7%	-30.3%
bs=100 sw=10 sl=64	MB/s	0.495 MB/s	0.531 MB/s	0.711 MB/s	-6.8%	-30.4%
bs=100 sw=10 sl=64	p50	118,902 us	113,332 us	91,298 us	+4.9%	+30.2%
bs=100 sw=10 sl=64	p95	160,971 us	140,624 us	98,241 us	+14.5%	+63.9%
bs=100 sw=10 sl=64	p99	160,971 us	140,624 us	113,458 us	+14.5%	+41.9%
bs=1000 sw=10 sl=64	throughput	958 tuples/sec	963 tuples/sec	1,197 tuples/sec	-0.5%	-20.0%
bs=1000 sw=10 sl=64	MB/s	0.585 MB/s	0.588 MB/s	0.731 MB/s	-0.5%	-19.9%
bs=1000 sw=10 sl=64	p50	1,026,719 us	1,032,597 us	899,658 us	-0.6%	+14.1%
bs=1000 sw=10 sl=64	p95	1,204,331 us	1,088,855 us	941,120 us	+10.6%	+28.0%
bs=1000 sw=10 sl=64	p99	1,204,331 us	1,088,855 us	990,298 us	+10.6%	+21.6%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,613.34,200,128000,326,0.199,30704.20,39342.72,39342.72
1,100,10,64,20,2463.81,2000,1280000,812,0.495,118901.92,160970.53,160970.53
2,1000,10,64,20,20881.72,20000,12800000,958,0.585,1026718.71,1204330.72,1204330.72

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

1 open finding

• The previous RC workflow explicitly bundled licenses/LICENSE-CC-BY-4.0.txt into the Docker… · New

What changed in this PR

This PR aims to make release artifacts and container images reproducible so rebuilding the same commit yields byte-identical outputs across environments.

Changes:

• Make frontend build numbers deterministic via SOURCE_DATE_EPOCH and remove Sass randomness affecting CSS output.
• Pin Docker base images by digest, introduce snapshot-based apt/pip installs, and clamp image timestamps during builds/pushes.
• Make RC tarball + Docker Compose bundle generation reproducible and document reproduction steps (incl. Renovate support for digest refreshes).

File	Description
frontend/​src/​build-version.spec.ts	Adds tests for SOURCE_DATE_EPOCH-derived build numbers and reproducibility.
frontend/​src/​app/​workspace/​component/​code-editor-dialog/​code-editor.component.scss	Removes nondeterministic Sass random() to stabilize CSS output.
frontend/​build-version.js	Implements deterministic build number derivation from SOURCE_DATE_EPOCH and exports it.
frontend/​build-version.d.ts	Adds typings for the new defaultBuildNumber API.
file-service/​src/​main/​resources/​rustfs-config.yml	Pins RustFS image tag to stable 1.0.0.
file-service/​src/​main/​resources/​docker-compose.yml	Pins RustFS image tag to stable 1.0.0.
file-service/​build.sbt	Replaces play-json milestone dependency with stable 3.0.6.
file-service/​LICENSE-binary	Updates binary NOTICE entries for the stable Play JSON jars.
config-service/​build.sbt	Replaces play-json milestone dependency with stable 3.0.6.
config-service/​LICENSE-binary	Updates binary NOTICE entries for the stable Play JSON jars.
common/​workflow-core/​src/​test/​scala/​org/​apache/​texera/​service/​util/​RustFSContainer.scala	Updates test container image tag to 1.0.0.
bin/​y-websocket-server/​Dockerfile	Pins Node image patch version and switches to npm ci for reproducible installs.
bin/​single-node/​docker-compose.yml	Pins several images (RustFS/nginx/alpine/curl) for reproducibility.
bin/​k8s/​values.yaml	Pins RustFS Helm chart image tag to 1.0.0.
bin/​k8s/​values-development.yaml	Pins RustFS Helm chart image tag to 1.0.0.
bin/​k8s/​templates/​base/​workflow-computing-unit-manager/​workflow-computing-unit-manager-deployment.yaml	Pins curl initContainer image to a concrete version.
bin/​k8s/​templates/​base/​lakefs/​lakefs-setup-job.yaml	Pins curl image to a concrete version.
bin/​k8s/​Chart.yaml	Pins RustFS chart dependency version to 1.0.0.
bin/​dockerfiles/​workflow-computing-unit-managing-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​dockerfiles/​workflow-compiling-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​dockerfiles/​texera-web-application.dockerfile	Pins base images by digest, uses snapshot installer, and switches Yarn install to immutable.
bin/​dockerfiles/​snapshot/​test_install.sh	Adds regression tests for snapshot installer logic (apt/pip split/cutoff).
bin/​dockerfiles/​snapshot/​pip-bootstrap-requirements.txt	Pins a pip version used to bootstrap --uploaded-prior-to support.
bin/​dockerfiles/​snapshot/​install.sh	Introduces snapshot-based apt/pip installation logic for reproducible dependency resolution.
bin/​dockerfiles/​notebook-migration-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​dockerfiles/​mounter.dockerfile	Pins Debian base by digest and routes apt installs through snapshot installer.
bin/​dockerfiles/​jupyter.dockerfile	Pins Jupyter base image by digest.
bin/​dockerfiles/​file-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​dockerfiles/​config-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​dockerfiles/​computing-unit-worker.dockerfile	Pins base images by digest and uses snapshot installer; removes silent fallback installs.
bin/​dockerfiles/​computing-unit-master.dockerfile	Pins base images by digest and uses snapshot installer for apt/pip.
bin/​dockerfiles/​agent-service.dockerfile	Pins bun image by digest and removes date-dependent bun cache + /etc/shadow variability.
bin/​dockerfiles/​access-control-service.dockerfile	Pins base images by digest and uses snapshot installer + SOURCE_DATE_EPOCH args.
bin/​build-images.sh	Passes reproducibility args and uses rewrite-timestamp=true registry outputs.
bin/​README.md	Documents reproducible image build inputs and manual rebuild commands.
.gitignore	Un-ignores bin/y-websocket-server/package-lock.json to support npm ci.
.github/​workflows/​create-release-candidate.yml	Uses reproducible tarball script from tag and threads IMAGE_TAG into email templating.
.github/​workflows/​build.yml	Updates RustFS container/image references to 1.0.0 and macOS artifact URL.
.github/​workflows/​build-and-push-images.yml	Derives and passes SOURCE_DATE_EPOCH/PACKAGE_SNAPSHOT and enables timestamp rewriting on push.
.github/​scripts/​test_create_release_tarballs.sh	Adds regression tests ensuring tarballs are byte-identical and metadata is normalized.
.github/​scripts/​create-release-tarballs.sh	New reproducible tarball + compose bundle generator used by RC workflow and voters.
.github/​renovate.json5	Enables Dockerfile manager and groups digest/tag bumps for pinned base images.
.github/​release/​vote-email-template.md	Documents how voters can reproduce tarballs and adds a checklist item.

🧠 Review effort: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[parshimers]

I'm waiting on #9072 to be fixed before rebasing and adding backport tags, that is what is causing the CI chaos here. I don't think it's related.

[mengw15]

Hi @parshimers, What is the status of this PR?

[parshimers]

Was blocked by the CI chaos but now it should be fine. Let's see if it passes now.

[mengw15]

Left some comments

[parshimers]

Good points, addressed hopefully. Fingers crossed on CI.

[mengw15]

LGTM

[parshimers]

This front-end flake is very tiresome... a huge waste of resources

[github-actions]

Backport PR opened: draft #9235 (#9235) to release/v1.3, assigned to @parshimers — needs manual work because the cherry-pick conflicts.