Spark-Hermes

One pinned agent. A competition on how it is instructed. Every result checkable by a stranger. SN74 Gittensor.

A strategy is prose — a SOUL.md and optional skills — and nothing else. The validator runs the same pinned Hermes agent and model under every strategy, in a sealed sandbox, on task instances whose grading criteria were committed before submissions opened. A strategy is paid for beating the strategy-less baseline on the same instances. The crowned strategy is merged as the incumbent; every verified trajectory is exported as training data.

• Live: https://gittensor-model-hub.github.io/Spark-Hermes/live/
• Submit: submissions/README.md
• Pins: docs/pins.md · Findings: docs/spikes.md
• Training data (public): gittensor-model-hub/spark-hermes-rounds

Layout

Path	What
sh/validator/	the round loop (orchestrate.py), episode runner and sealed sandbox, grader, inference proxy, batch runner
sh/scoring/	v2.py — payment: paired delta against the baseline over the pooled window, one-sided lower bound; crown.py — the crown, on this round alone
sh/predicates/	the closed predicate vocabulary tasks are graded with; copied into the runner image at build
sh/exports/	the king's SFT rows and DPO pairs from a closed round, leak scan, Hugging Face upload
sh/cli/	miner (tasks / submit / status), lint (the contract CI runs), attest (the hotkey's signature), scorecard, mock_miners
sh/web/	the static page for a closed round
submissions/	one directory per hotkey, prose only — merged when crowned
rounds/	every round's tasks (what miners were shown), window, seal, close.json, crown.json, reveal.json, checks, scorecards, and evaluated/ — the scored tasks, when miners were shown previews; queue.json — digests of rounds minted ahead
docs/	the site served by Pages: landing, live dashboard, round pages; pins and findings

Task families and their withheld halves live in a private repository; only what a round publishes is here.

Compete

A round opens with a 2-hour submission window; the board shows the countdown. The family is swe_fix: fix a real bug in a real Python repository. While the window is open you get 6 practice bugs — one per hidden bug, from the same repository and environment as that bug (a repository can appear twice) — and the round is scored on 6 different, hidden bugs from those same repositories, published when it closes.

python -m sh.cli.miner tasks                                   # this round's practice bugs; nothing else exists to fetch
$EDITOR my-strategy/SOUL.md                                    # prose for debugging those codebases
python -m sh.cli.miner submit --bundle my-strategy --key ~/.bittensor/wallets/<cold>/hotkeys/<hot> \
        --checkout . --head-owner <your github user>           # one signed PR per hotkey; resubmit to replace

When the window closes the validator seals every strategy PR at its head SHA, evaluates, and posts the scorecard on the PR. The best Δ vs baseline on this round's instances is crowned and merged; the rest are closed; the next round opens at once. Payment pools the last 8 rounds. Details: submissions/README.md.

Run the validator

uv sync --extra dev
SH_SALT_SECRET=... DOCKER_HOST=ssh://<worker> python -m supply.queue --queue queue --plan swe_fix:8:1 --ahead 3 --screen --screen-band 0.0:0.99   # private repo: mints rounds ahead, images built on the worker
HF_TOKEN=... uv run python -m sh.validator.orchestrate --queue ../Spark-Hermes-Withheld/queue       # forever; --once for one round
uv run python -m sh.validator.orchestrate --help

The control plane (credentials, GitHub, Hugging Face) stays on the machine you run this on; episodes run on a GPU worker over ssh and rsync, inside sh/validator/images/ built at the pinned Hermes commit. The secrets — HF_TOKEN for uploads, GH_TOKEN for the loop's GitHub identity, SH_SALT_SECRET for the queue daemon — are read from the environment and never from a file. A restarted loop resumes an unfinished round from the stage it reached and never changes what that round sealed.

Check a round

rounds/<id>/reveal.json carries every withheld half and salt. For each instance, "hmac-sha256:" + HMAC(salt, canonical_json(withheld)) must equal the commitment in the task record — in rounds/<id>/tasks/ when miners were shown the evaluated tasks, in rounds/<id>/evaluated/ when they were shown previews — and rounds/queue.json, published before the round opened, carries the digest of both. For swe_fix, reveal.json also names each scored bug's SWE-smith instance id: the committed test ids are that bug's own tests, readable in the public dataset. close.json recomputes from the graded episodes. Nothing here needs to be trusted.

Develop

uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest -q

MIT. Preserve the upstream licenses of the pinned model and of Hermes Agent.