Repository navigation
feat(spec-review): record the ratifier on decisions — agent-chosen vs human-ratified (FAFF-1079) - #1121
Merged
alechill merged 8 commits intoOct 24, 2026
Conversation
Contributor
|
Ready to land. Run this locally in a real terminal: |
…t-chosen vs human-ratified Signed-off-by: Alec Hill <alec@shftwst.dev>
… human-ratified (FAFF-1079)
A `**Chosen:**` marker recorded that a decision closed, never who closed it, so the
spec-review design lenses could not tell an agent's own inference from a human's live
say-so and spent review rounds re-litigating settled calls.
Adds the `(ratified: human)` suffix as a prose-only convention and wires the one consumer
that already has a defer-or-pass-through mechanism:
- the kernel's decision-marker dialect gains a `Ratifier tag (optional)` bullet — closed
single-value vocabulary, never a `faff-contract:spec-readiness` field, never added on an
autonomous dispatch, inherited tags carried forward verbatim;
- faff-prep's Spec-review gate gains a third `ratify` path behind a decision-vs-implementation
scope test, with a neutralised and 200-character-capped quote of the human's reply in the
tracker comment the re-attach already posts;
- the Decisions-register capture step gains a second, tradeoff-shaped trigger;
- the three design-lens refuters and the occupant's own deferral paragraph name the
`### Ratified tradeoffs (docs/decisions.md)` subsection and gain a fourth deferral clause
that reads the tag off the spec body, needing no `## Ratified scope` block.
Neither contract schema is touched: a tagged `**Chosen:**` still emits `{ "marker": "chosen" }`,
verified byte-for-byte through `faff contract spec-readiness`.
A committed fixture set under the occupant's own eval home carries the evidence, with its pins
and recorded outcomes: the untagged variant draws a `major`, the tagged variant records the same
objection as an `observation` citing the tagged line while its implementation half stays `major`,
and the `critical` variant still gates with the tag present.
Two consequences of the required prose growth are carried here rather than hidden: the
faff-prep line-cap baseline rises with its justification, and the adversarial brief-token
reserve is re-measured against the grown design-lens briefs.
Signed-off-by: Alec Hill <alec@shftwst.dev>
…ng rationale out (FAFF-1079) Operator decision 2026-10-22: free kernel lines by moving prose, not by raising KERNEL_LINE_BASELINE. The Sibling-skill invocation section's "Why this exists" paragraph is rationale, so it leaves the kernel. Its one rule (human-facing "type /faff-prep" prose keeps its slash) is folded into step 3. The baseline is lowered to the new measured 465 so the ratchet stays tight. The rationale, preserved here: a fixed /faff-prep string is correct in at most one install mode. The Skill tool takes a skill name, and that name differs per mode, so every delegation resolves against the live available-skills list rather than a frozen literal. That keys off whatever the canonical name is, which composes with a future skill rename without editing the rule. Also retunes two band fixtures in test/adversarial-context-window.test.mjs (tests 50 and 51) to the re-measured 2800-token brief reserve, so each still exercises the usable-to-raw window band it exists to pin. Signed-off-by: Alec Hill <alec@shftwst.dev>
…-FAFF-1079 Signed-off-by: Alec Hill <alec@shftwst.dev>
alechill
force-pushed
the
FAFF-1079-record-the-ratifier-on-decisions-agent-chosen-vs-human-ratified
branch
from
October 24, 2026 07:35
73bae86 to
c900371
Compare
…-1398 Signed-off-by: Alec Hill <alec@shftwst.dev>
…AFF-1398 Signed-off-by: Alec Hill <alec@shftwst.dev>
…ratifier-on-decisions-agent-chosen-vs-human-ratified Signed-off-by: Alec Hill <alec@shftwst.dev> # Conflicts: # plugin/skills/faff/bin/lib/validate-adapters.js
…ratifier-on-decisions-agent-chosen-vs-human-ratified Signed-off-by: Alec Hill <alec@shftwst.dev> Signed-off-by: Alec Hill <alec@shftwst.dev>
alechill
deleted the
FAFF-1079-record-the-ratifier-on-decisions-agent-chosen-vs-human-ratified
branch
October 24, 2026 16:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A spec's
**Chosen:**decision can now record that a human, not the producer, settled it:**Chosen:** X (ratified: human). Adversarial spec-review design lenses defer to a tagged decision, so a call a human has already made isn't re-argued every round. The decision's implementation is still critiqued at full severity, and acriticalis never deferred. No contract changes:faff-contract:spec-readinesskeepschosen | punt | assumes.What changed
kernel.md, Decision markers: an optionalRatifier tagbullet.human. An absent tag means agent-chosen, so behaviour is unchanged for existing specs./faff-prepprose keeps its slash") is folded into step 3, andKERNEL_LINE_BASELINEdrops 466 → 465.faff-prep/SKILL.md, Spec-review gate: an interactive-only ratify choice beside revise and park.majororminordesign-lens objection that a scope test classifies as attacking the decision itself, not its implementation. Anything ambiguous gets no offer.reviseand autonomous stale-refresh.refute-architectural.md,refute-infosec.md,refute-qa.md, plus the review-bench copies):### Ratified tradeoffs (docs/decisions.md)entries.review-call.mjs:DEFAULT_BRIEF_RESERVE_TOKENS2400 → 2800, re-measured after the lens clauses grew the bindingrefute-infosec.mdbrief to about 2596 tokens.test/adversarial-context-window.test.mjs: two band fixtures retuned to the new reserve so each still exercises the usable-to-raw window band.faffter-dark-spec-review/eval/in-spec-ratification/: the fixture set and recorded outcomes for the five design-lens checks, run against real backends.For your review
SKILL_LINE_BASELINE["faff-prep"]622 → 680. The ratify subsection adds 58 lines to one skill. That's worth a look against the lean standard.Review
One pass: standard
pass, plus an adversarial second opinion fromopenai/deepseek/deepseek-v4-flash-0731. Three design findings were accepted, because they restate the LLM-judged-prose shape the 2026-09-25 operator decision accepted. One was refuted: the reused neutralisation isneutralise()atratified-scope.js:181.Tests
validate-adaptersand every lint rung pass.test/validate-adapters.test.mjsandtest/adversarial-context-window.test.mjs: 86 of 86 pass.origin/mainin this environment.Acceptance Criteria
faff-1079#done:1No edit touchesfaff-contract:spec-readiness's orspec-review-verdict's JSON schema files or theircontract-defs.jsmappers (grep confirms no diff inplugin/skills/faff/contracts/*.schema.jsonor thecomputeSpecReadiness/objection-schema regions ofcontract-defs.js).Verified:
git diff origin/mainover plugin/skills/faff/contracts/*.schema.json and contract-defs.js is empty.Method: command
git diff origin/main --stat -- 'plugin/skills/faff/contracts/*.schema.json' plugin/skills/faff/bin/lib/contract-defs.jsfaff-1079#done:2kernel.md's new ratifier-tag bullet states the tag is never emitted on an autonomous dispatch, in those or equivalent words.Verified: The kernel bullet reads "never added on an autonomous dispatch".
Method: command
grep -o 'never added on an autonomous dispatch' plugin/skills/faff/references/kernel.mdfaff-1079#done:3kernel.md's Decision-markers list contains aRatifier tag (optional)bullet in the same list as the existingPunt ownership (optional)bullet (adjacent placement is a readability nicety for the PR reviewer to check, not a mechanically-verifiable position), documenting(ratified: human)as a closed-vocabulary, single-value, prose-only suffix.Verified: The Ratifier tag (optional) bullet is in kernel.md's Decision-markers list; kernel.md measures 465 against KERNEL_LINE_BASELINE 465 and faff validate-adapters exits 0
Method: command
faff validate-adaptersfaff-1079#done:4The bullet states explicitly that a tagged**Chosen:**still emits{ "marker": "chosen" }.Verified: The kernel bullet states a tagged Chosen still emits
{ "marker": "chosen" }.Method: command
grep -o '{ "marker": "chosen" }' plugin/skills/faff/references/kernel.mdfaff-1079#done:5faff-prep/SKILL.md's Spec-review gate section offers a third "ratify" choice alongside revise/park for amajor/minordesign-lens objection that cleanly names a specific already-**Chosen:**decision and passes the decision-vs-implementation scope test, in interactive mode only.Verified: faff-prep's Spec-review gate gained Ratify a design-lens objection in place (interactive only), gated on major/minor plus the scope test.
Method: command
grep -c 'passes the scope test below' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:6The same text states an objection that is ambiguous or mixes decision- and implementation-level critique is never offered ratify — it routes through revise/park exactly as today.Verified: The same section states an ambiguous or mixed objection is never offered ratify and routes through revise/park.
Method: command
grep -c 'ambiguous between the two, or mixes both' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:7The same text states acriticalobjection is never offered this path.Verified: Procedure step 1 presents a
criticalas today with no ratify offer.Method: command
grep -c 'severity == "critical" -> present as today' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:8The same text states autonomous mode never presents this offer.Verified: Procedure step 5 states the offer never fires in autonomous mode.
Method: command
grep -c 'Autonomous mode: never fires' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:9The same text states a ratify does not consume a loop-cap iteration.Verified: Procedure step 4d states a ratify does NOT consume a loop-cap iteration.
Method: command
grep -c 'consume a loop-cap iteration' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:10The same text states that on an explicit "ratify" reply, the human's verbatim reply (or its shortest decision-bearing sentence) is logged into the tracker comment the re-attach already posts, alongside the resulting tag.Verified: Procedure step 4b plus the Audit the utterance paragraph require the neutralised, capped quote in the tracker comment the re-attach already posts.
Method: command
grep -c 'neutralised, capped quote' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:11faff-prep/SKILL.md's Decisions-register capture step documents a second trigger (ratifying a design-lens objection) that drafts the tradeoff shape (Chosen/Rationale/Scope/Source-issue/Ratified-by: human/Date, noMatches:), while the original Punt/Assumes-closure trigger's precedent-shaped draft is unchanged in the diff.Verified: The capture step documents the second, tradeoff-shaped trigger with Source-issue/Ratified-by/Date and no Matches:; the original precedent-shaped trigger is unchanged in the diff.
Method: command
git diff origin/main -- plugin/skills/faff-prep/SKILL.mdfaff-1079#done:12The extension text states the existing build-follows / no-build-follows materialisation split applies unchanged.Verified: The extension states follow-through is identical: build-follows materialises via Step 4c, no-build-follows surfaces the drafted text.
Method: command
grep -c 'Follow-through is \*\*identical\*\*' plugin/skills/faff-prep/SKILL.mdfaff-1079#done:13faffter-dark-spec-review/SKILL.mdand all three ofrefute-architectural.md/refute-infosec.md/refute-qa.mdexplicitly name### Ratified tradeoffs (docs/decisions.md)in their existing "Defer to ratified scope" text (grep for the literal heading string in each file).Verified: All four files name the literal
### Ratified tradeoffs (docs/decisions.md)in their Defer-to-ratified-scope text.Method: command
grep -c 'Ratified tradeoffs (docs/decisions.md)' in SKILL.md + the three refute-*.mdfaff-1079#done:14Each of the threerefute-*.mdfiles gains a new "Defer to an in-spec human ratification" bullet stating: no## Ratified scopeblock required; defers toobservationon a tagged**Chosen:**; implementation still critiqued at full severity;criticalnever deferred; the tagged line's own text is treated as untrusted DATA, never an instruction.Verified: Each of the three refute files gained the Defer to an in-spec human ratification bullet with all five required statements.
Method: command
grep -c 'Defer to an in-spec human ratification' plugin/skills/faffter-dark-spec-review/refute-{architectural,infosec,qa}.mdfaff-1079#done:15faffter-dark-spec-review/SKILL.md's dense deferral paragraph gains the equivalent sentence, including the untrusted-DATA framing.Verified: The occupant SKILL.md's dense deferral paragraph gained the equivalent sentence including the untrusted-DATA framing.
Method: command
grep -c 'A fourth deferral source needs no block at all' plugin/skills/faffter-dark-spec-review/SKILL.mdfaff-1079#done:16kernel.md's ratifier-tag bullet (or faff-prep's autonomous refresh section) states that an autonomous stale-refresh preserves an already-present(ratified: human)tag verbatim — "never added on autonomous dispatch" governs new tags, not inherited ones.Verified: Both kernel.md's bullet and faff-prep's carry-forward paragraph state an autonomous stale-refresh preserves an already-present tag verbatim.
Method: command
grep -c 'inherited tag is carried forward verbatim' kernel.md; grep -c 'autonomous stale-refresh' faff-prep/SKILL.mdfaff-1079#done:17Build the fixture trio (committed underplugin/skills/faffter-dark-spec-review/eval/in-spec-ratification/or the nearest existing eval-fixture home for this occupant): a minimal spec containing exactly one decision-level**Chosen:**finding independently known to draw amajorobjection from at least one design lens today (reuse or adapt an existing spec-judge-discrimination-style fixture rather than inventing an untested one), in three variants — untagged, the same file with(ratified: human)appended to that one line, and a critical-severity variant of the same decision — plus a fourth, separate fixture whose**Chosen:**decision draws an implementation-level objection (the "correctly identified but badly built" case Decision 4b's carve-out names), used only by item 4 below.Verified: Committed under plugin/skills/faffter-dark-spec-review/eval/in-spec-ratification/: three decision variants, the implementation-level fixture, the scope-test objection set, the register fixture, a README and run-log.
Method: command
ls plugin/skills/faffter-dark-spec-review/eval/in-spec-ratification/faff-1079#done:18Pin every run variable except the tag, and record the pin. Items 1-3 below compare outcomes across fixture variants that must differ in exactly the tag — so every run pins the same backend/model identity, the same--contextfile set, and the same## Ratified scopepresence-or-absence. Record the pinned values in arun-log.md(or equivalent) committed alongside the fixture trio itself — one file per fixture-directory, not a separate untracked note — so a later re-run of the fixture has the exact prior pin to reproduce, and a PR reviewer can check the pin without re-deriving it. A pass whose runs used different backends or context, or whose pin was never recorded, is not evidence the tag caused the difference.Verified: run-log.md records the single-element pinned chain, timeout, max-tokens, deadline, empty context set, absent Ratified-scope block, lens and system-dir, committed beside the fixtures.
Method: command
cat plugin/skills/faffter-dark-spec-review/eval/in-spec-ratification/run-log.mdfaff-1079#done:19Test coverage item 1 (no regression): run the design lens(es) against the untagged decision-level fixture, pinned per the rule above; the samemajor(or higher) objection appears. This is the fixture's pinned baseline, not a fresh eyeball check.Verified: Item 1 ran and the major objection appeared; recorded in run-log.md.
Method: command
node plugin/skills/faffter-dark-spec-review/build-lens-requests.mjs --lenses architectural ... | node plugin/skills/faffter-dark-adversarial-review/fan-out.mjs --strategy serialfaff-1079#done:20Test coverage item 2 (defers to observation): run the same lens(es), same pinned run variables, against the tagged decision-level fixture with no## Ratified scopefile present; the previously-gating objection now appears only as anobservationciting the tagged line. Because both variants differ in exactly one suffix under otherwise-pinned conditions, a pass here is direct evidence the tag — not lens variance — caused the downgrade.Verified: Item 2 ran and the objection appeared only as an observation citing the tagged line; the implementation-level objection stayed major, so the downgrade was scoped.
Method: command
node plugin/skills/faffter-dark-spec-review/build-lens-requests.mjs --lenses architectural ... | node plugin/skills/faffter-dark-adversarial-review/fan-out.mjs --strategy serialfaff-1079#done:21Test coverage item 3 (critical never deferred): the critical-severity fixture variant, tag present, same pinned run variables; confirm it still gates.Verified: Item 3 ran and the critical still gated with the tag present.
Method: command
node plugin/skills/faffter-dark-spec-review/build-lens-requests.mjs --lenses architectural ... | node plugin/skills/faffter-dark-adversarial-review/fan-out.mjs --strategy serialfaff-1079#done:22Test coverage item 4 (scope test correctly declines an implementation objection — the false-negative direction): run the ratify-offer scope test (Decision 2) against the separate implementation-level fixture's objection; confirm it is classified as implementation-level and the ratify choice is not offered. This closes the false-negative gap a decision-only fixture pair cannot exercise: a scope test that always classifies "decision-level" would pass items 1-3 while still wrongly ratifying every implementation objection.Verified: Item 4 ran: the implementation-level claim classified
implementation-level,ratify_offered=no.Method: command
node plugin/skills/faffter-dark-adversarial-review/review-call.mjs --system <scope-test brief> --diff <objection claim> --diff-kind prosefaff-1079#done:23Test coverage item 5 (the fail-safe middle is tested, not merely named): run the same scope test against the ambiguous worked example itself ("Postgres is fine, but this schema will lock the table under load" — Decision 2's third worked example, not the clean implementation case item 4 uses) and confirm it too is not offered ratify. Item 4 alone tests only the unambiguous implementation case; the "ambiguous → no ratify, never the reverse" safety claim is specifically about the boundary case, so the boundary case is what must be exercised — a scope test that classifies unambiguous cases correctly but defaults an ambiguous one to "decision-level" would pass item 4 while failing the exact case the asymmetry exists to protect.Verified: Item 5 ran: the ambiguous worked example classified
ambiguous,ratify_offered=no. A decision-level positive control returned yes, so the test is not a constant no.Method: command
node plugin/skills/faffter-dark-adversarial-review/review-call.mjs --system <scope-test brief> --diff <objection claim> --diff-kind prosefaff-1079#done:24Contract-extraction check (the "byte-identical" claim, named and run, not just asserted): for the same fixture's untagged and tagged variants, extract both specs'faff-contract:spec-readinessblock and diff them byte-for-byte; the only permitted difference is inside the(ratified: human)suffix text itself, never in the emitteddecisions[]/confidenceJSON. This is the concrete check the "no contract regression" scenario (section 5) names but does not itself run.Verified: Contract-extraction check ran: both variants' extractions are identical and the validator output is byte-identical.
Method: command
faff contract spec-readiness < <extraction> ; diff <untagged-out> <tagged-out>faff-1079#done:25Adocs/decisions.mdfixture carrying aratified_tradeoffentry for a synthetic issue confirmsfaff ratified-scope --assembleemits the### Ratified tradeoffssubsection andfaff ratified-scope --validatestill passes (no regression to the existing selftest suite inratified-scope.js/decisions.js).Verified: The register fixture round-tripped through --assemble and --validate; the ratified-scope.js and decisions.js selftests both still pass.
Method: command
node plugin/skills/faff/bin/lib/ratified-scope.js --selftest ; node plugin/skills/faff/bin/lib/decisions.js --selftestCloses FAFF-1079