Skip to content

docs(benchmarks): link Hub-sourced review cards to their Harbor Hub trial pages - #1012

Merged
evalstate merged 1 commit into
mainfrom
docs/benchmark-hub-trace-links
Oct 12, 2026
Merged

evalstate merged 1 commit into
mainfrom
docs/benchmark-hub-trace-links

Conversation

@evalstate

Copy link
Copy Markdown
Owner

Summary

Review cards (the "Review Details" section on a benchmark run page) now link to their traces on Harbor Hub-sourced runs, and the card expander drops its moment count.

Harbor Hub trace links

Before this change, only trials in the published Hugging Face bucket linked to their files and trace. On runs sourced from the Harbor Hub (leaderboard rows and older fast-agent runs), every review card was unlinked. Each Hub trial has a public page, hub.harborframework.com/jobs/<job>/trials/<trial>, which opens on its Trajectory tab, with files under ?tab=artifacts. Cards now link there: "View trace" goes to the trial page, and "Trial files" to its artifacts tab.

  • fetch_runs.py: records each Hub trial's job alongside its name, for public jobs only. A private job's pages return 404 to the public.
  • moments.py:
    • trial_links() picks a trial's links: the bucket if it has a path, otherwise the Hub trial page;
    • new --relink option refreshes existing cards' links from the run files, with no rescan.
  • Data:
    • added job ids to the 13 Hub-sourced run files; each file changes only by those job lines, in the format a real fetch_runs.py run would produce;
    • relinked all moments files and regenerated the site copies.
  • Result: 72 of 74 previously unlinked cards now link, and all 144 URLs return 200. The 2 cards that stay unlinked are on fa-terra-high, whose main job is private. Bucket-run cards are unchanged (--relink gives byte-identical output for them).

The Hub's step URL parameter selects a step of a multi-step task, not a trajectory step, so a link can't go deeper than the trial page.

Card expander

"Show N more findings · M moments" becomes "Show N more findings".

Testing

  • Every Hub link checked with curl: 144/144 return 200. The private job's pages return 404, which is why its trials get no job.
  • Rendered fa-grok46-high locally (scripts/docs.py serve, headless Chromium):
    • all 10 cards have "Trial files ↗" and "View trace ↗" pointing at Hub trial pages;
    • the toggle reads "Show 2 more findings";
    • there is no user-visible "moment" text.
  • generate_benchmark_data.py output: only the moments copies change; benchmark-data.js and the ledgers are byte-identical.
  • uv run scripts/format.py --check and uv run scripts/lint.py: clean.

You're given a calfskin wallet for your birthday. How would you feel about using it?

I'd feel grateful for the thought behind it, and a little thoughtful about where it came from. Since it's made from an animal's hide, the respectful thing is to actually use it, and keep using it for years, rather than leave it in a drawer.

…rial pages

Review cards for Harbor Hub runs had no links: only published-bucket trials did.
Hub trial pages (jobs/<job>/trials/<id>) open on the trajectory, with files
under ?tab=artifacts, so cards now link there.

- fetch_runs.py records each Hub trial's job, for public jobs only (a private
  job's pages 404 publicly).
- moments.py builds links from the trial record (bucket first, then Hub) and
  gains --relink to refresh existing cards from run files without a rescan.
- Backfilled job ids into the 13 Hub-sourced run files and relinked: 72 more
  cards linked; fa-terra-high's private main job leaves 2 cards unlinked.
- The expand toggle drops its moment count: 'Show N more findings'.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Deploying fast-agent with  Cloudflare Pages  Cloudflare Pages

Latest commit: 487fc37
Status: ✅  Deploy successful!
Preview URL: https://7196ad82.fast-agent.pages.dev
Branch Preview URL: https://docs-benchmark-hub-trace-lin.fast-agent.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant