Skip to content

fix(bench): send a User-Agent when ingest downloads results - #2321

Merged
0xAndoroid merged 1 commit into
mainfrom
fix/bench-ingest-download
Nov 6, 2026
Merged

0xAndoroid merged 1 commit into
mainfrom
fix/bench-ingest-download

Conversation

@0xAndoroid

@0xAndoroid 0xAndoroid commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The benchmark ingest script now sends a descriptive User-Agent header on every download, because Cloudflare rejects Python's default one. The ingest script (scripts/bench/ingest.py) downloads a finished benchmark result and publishes it. In ingest run 42153703631, the result host's Cloudflare Browser Integrity Check answered the download with HTTP 403 (error code: 1010), the code Cloudflare returns when it blocks a client by its browser signature.

All ingest downloads go through one function, download(). It now sets User-Agent: jolt-e2e-ingest/1 (+https://github.com/a16z/jolt) on each HTTPS request, both for public result URLs and for authenticated GitHub artifact requests. When a request redirects, the header stays on the redirected request, and the Authorization header is still removed.

I checked the header against the live host with a result URL that names a result that does not exist. Python's default User-Agent got HTTP 403, the Cloudflare block, and the new User-Agent got HTTP 404, the host's own not-found answer, so the new header passes the Cloudflare check. Not yet verified: a successful download of a real result. That needs a rerun of the calibration benchmark whose ingest failed; a calibration run benchmarks one main revision against itself on the benchmark host to measure noise for later PR comparisons.

A separate failure was fixed on the host without a code change. The first failed calibration attempt left the cached Git checkout on the benchmark host, jolt-bench, with a commit whose parent object was missing but was not recorded as a shallow boundary in .git/shallow. The next run's checkout walked from that commit into the missing parent and failed with internal error in revision walk. Fetching that commit again with --depth=1 recorded the boundary and repaired the cache once. Current checkout code already fetches with --depth=1, so this PR changes no Git handling.

Changes

  • Add one regression test covering public and authenticated downloads, the User-Agent on redirected requests, and removal of Authorization on redirects.

Testing

All 14 tests passed with:

uv run --no-project python -m unittest scripts.tests.test_e2e_ingest scripts.tests.test_e2e_publication -v

Rust checks were not run because no Rust or Cargo files changed.

Security Considerations

HTTPS validation and redirect credential removal are unchanged. The proof and verifier code are unchanged.

Breaking Changes

None.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

📏 PR diff

Note

Actual code changed: 0 lines

🟢 0 added    🔴 0 removed

Tests, docs, fixtures, and helper scripts excluded.

Category 🟢 Added 🔴 Removed
⌨️ Actual code 0 0
🧪 Tests +15 0
📚 Docs 0 0
📦 Fixtures 0 0
🔧 Helper scripts +2 −1
Total diff +17 −1

Every changed line is classified once.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Verifier classifier: risk-1

Head 3c19429ed2ad60e616c9004e3727463cb106999a

In both revisions, the changed downloader serves only benchmark CI ingestion and publication. Adding a User-Agent header cannot reach Jolt verifier acceptance.

Risk: risk-1

  • scripts/bench/ingest.py: risk-1 — Only adds an HTTP User-Agent to benchmark artifact downloads; this CI/reporting script has no production verifier effect.
  • scripts/tests/test_e2e_ingest.py: risk-1 — Adds mocked tests for the User-Agent and redirect behavior; no production behavior changes.

gpt-6.1-sol, reasoning effort xhigh

@github-actions github-actions Bot added the risk-1 Tests, scripts, CI or documentation only label Nov 6, 2026
@0xAndoroid 0xAndoroid added the pika-keep-healed Pika auto-heals this PR: merges main on conflict, fixes failing CI (never force-pushes) label Nov 6, 2026
@0xandoroid-ai-agent

Copy link
Copy Markdown

This pull request #2321 has been reviewed at commit 3c19429 using skill reviewer and model Claude Opus 5.5 with xhigh reasoning effort, and no issues were found.

@github-actions github-actions Bot added the merge-eligible AI merge gate review policy satisfied for the current head label Nov 6, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AI merge gate approves 3c19429ed2ad60e616c9004e3727463cb106999a. Review attested by 0xandoroid-ai-agent: An authorized comment attests a clean review of the current head 3c19429ed2ad60e616c9004e3727463cb106999a, so the gate approves it.

@0xAndoroid
0xAndoroid merged commit 428e805 into main Nov 6, 2026
48 checks passed
@0xAndoroid 0xAndoroid removed the pika-keep-healed Pika auto-heals this PR: merges main on conflict, fixes failing CI (never force-pushes) label Nov 6, 2026
@github-actions github-actions Bot removed the merge-eligible AI merge gate review policy satisfied for the current head label Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk-1 Tests, scripts, CI or documentation only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant