Skip to content

Human-decided routes: approval gates answered in the controller - #223

Open
wseaton wants to merge 15 commits into
mainfrom
human-decisions
Open

wseaton wants to merge 15 commits into
mainfrom
human-decisions

Conversation

@wseaton

@wseaton wseaton commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

A route can now be decided by a person: route(human = True, review = "...", timeout = "20m"). The engine opens a decision request with the controller carrying the evidence (dependency outputs, declared files, spend, what each answer starts, a rendered review), parks the route while independent tasks keep running, and polls until someone allowed run:approve answers or the request expires. Expiry answers uncertain, which the author routes like any other label.

The controller holds requests (one per run and route, digest-bound answers, audited), lists them on Approvals, and renders the evidence page with sandboxed HTML/SVG visualizations, Markdown, CSV and JSON. The web UI raises a browser notification when a request opens. Local runs authenticate with a per-run token.

Questions can be single choice, multi-select, or a pick from a list a dependency produced; over = gate.nodes fans out over the picked values. examples/human-gate runs the whole flow on a laptop against just controller-local.

The UI also gains an error boundary, so a page that throws shows the error instead of a blank window.

Changelog: Added: routes decided by a person, answered from the controller UI with the evidence, review and visualizations shown inline, and a browser notification when one opens

route(human = True) asks a person through the controller. The executor
opens the route's request and parks it: tasks that do not depend on the
route keep dispatching, and the route settles when the request is
answered (labels plus the decision record), expires (every question
uncertain), or the run halts. The request carries the evidence the
approver sees: dependency outputs and captured files, spend and
ceilings, the tasks each answer starts, and an optional review template
rendered with every value escaped for CommonMark.

The run reaches the controller on its ingest surface with the pod's
ingest credential; `human` is available exactly when that surface is.
Contract 1.15.0 adds the request types and the decision_wait event.

The RFC amendment specifies the human decider, its evidence, and
run:approve.

Assisted-by: Claude
The run opens and polls requests on its pod ingest surface; a local
run gets a credential minted at spawn for its local-<run> pod name,
revoked when the run completes. A person allowed run:approve lists open
requests, reads one with its evidence, and answers it against the
evidence digest; the first answer wins, a repeat is acknowledged, and an
accepted answer is audited in the same transaction. Requests still open
when their run completes are withdrawn.

run:approve defaults to the principal the run is attributed to and a
maintainer of an owning team, and a run principal is forbidden it.

Assisted-by: Claude
Approvals lists the open requests the signed-in user may answer, and
/decisions/:id shows one: the rendered review, each dependency's
output and files (images, CSV, Markdown, JSON, and HTML or SVG in a
sandboxed frame with no network), the tasks each answer starts, and the
answer controls. The shell raises a browser notification when a request
the user may answer opens; clicking it opens the request.

The decision DTOs take a Decision prefix: utoipa keeps the last schema
registered under a name, so EvidenceFileDto had silently replaced the
task-evidence schema.

examples/human-gate holds a priced training launch for approval and
runs locally against `just controller-local`.

Assisted-by: Claude
A human-decided route can now ask a choice that takes several labels
(`choice(multiple = True)`) and a pick whose options are a list a
dependency produced (`pick(source = plan.nodes)`). Picked values land
under the question id as a string list, so `over = gate.nodes` fans out
over what the approver chose. Models and output deciders still answer
one label; validation refuses picks and multi-select on them, a `when`
naming a pick, and a question named `decision`.

Answers are a list of values per question on the wire and in the
store. The controller and the engine both refuse values a question
does not offer, repeats, and more than one value on a single-answer
question.

Run-end cleanup no longer marks an already-expired request withdrawn;
expiry is computed on read, so the row still said open. The decision
store returns typed errors instead of anyhow::bail.

The UI gets a root error boundary and one around the routed page, so a
page that throws while rendering shows the error and a reload button
instead of a blank window.
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Docs preview for this PR is built and attached as the docs-preview artifact.

Download docs-preview, unzip, and open index.html in a browser.

Rebuilt for 819f563.

wseaton added 11 commits October 6, 2026 15:14
decide now works on plain inputs (each dependency's status and output, a captured-file lookup) and returns a Settled value; exec builds those inputs and turns Settled into a TaskResult. The modgraph check forbids the cycle.
runs::ingest_drop merged decisions::ingest_routes while decisions reads IngestState from it, a module cycle the modgraph check refuses. The server now merges both routers on the shared ingest state.
Resolves the overlap with the OpenAI decider (#226):

- QuestionKind keeps Score beside Choice { multiple } and Pick; Answer
  carries score, asked_as and labels. A person's score answer records
  its level position.
- The broker's new decide module refuses pick questions; plan
  validation already keeps them off model routes.
- needs = "decision" replaces "systemone" in the route checks.
- Routes may now map over a list, but a human route may not: one
  request per route. A human route may not declare files either; its
  evidence already carries them.
- The runner's captured_file hook is main's, keyed by producer name.
- CONTRACT_VERSION 1.17.0; the decision tables move to migration 0055.
Takes main's base64 0.23 beside the minijinja dependency in crucible/Cargo.toml.
Main took migration 0055 for rerun launches; the decision tables move to 0056. The local run environment keeps both the ingest variables and main's CRUCIBLE_RUN_NAME.
Main's run tokens (#243) moved playbook pods off the pod ingest surface:
a run now carries CRUCIBLE_CONTROLLER_URL, CRUCIBLE_RUN_NAME and a crr_
token bound to it. Decision requests follow:

- A run opens and polls its requests at /api/runs/{run}/decisions with
  its run token; a token reaches only its own run's requests. The pod
  routes, the local_run_tokens table and the local branch of ingest
  auth are gone.
- A local run gets a run token minted at spawn alongside the controller
  URL, and run_ended revokes the run's tokens.
- The engine's desk reads the same three variables the evidence client
  does.

Main also took migrations 0055 and 0056; the decision table is 0057.
getent exits non-zero when it cannot resolve a name, and under pipefail the J_HOST assignment then ended the pods shard with no message, before the gateway fallback or the script's own error ran. The gateway lookup's grep had the same failure. main's kind-e2e fails the same way.
Main fixed the same scenario JP host lookup in #245; its version of pods.sh replaces ours.
Main renamed run tokens to pod tokens (#246): CRUCIBLE_RUN_TOKEN is
CRUCIBLE_POD_TOKEN and grants name a Subject. Decision requests, the
local run's token and the engine's desk follow the new names.

Main took migration 0057 and contract 1.18.0; the decision table is
0058 and this branch's contract changes are 1.19.0. C-RUN-PRINCIPAL
keeps main's pod evidence routes wording and says decision requests use
the run's credential beside them.
Main took migration 0058 and contract 1.19.0 (#250); the decision table
is 0059 and this branch's contract changes are 1.20.0.
Main added lane admission against the cost ceiling (#251). A human
route asks a person, not a model, so it does not spend; ExecCfg keeps
both the decision desk and the lane width.

Pod evidence moved to /api/evidence (#249), so C-RUN-PRINCIPAL no
longer places the decision routes beside it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant