Repository navigation
Playbook: Demonstrate Repeatability: Replay, Recall, Rerun #133
Description
Repeatability governance (Rerun/Recall/Replay)
Immediate, low-risk scope (recommended first slice):
Fix the PlaybookWorkflowGraphExecutor bug: default metadata to PlaybookReplayMode.Rerun unless a replay context says otherwise.
Give every runner (SQL Statistics, Taxonomy, Essay) an explicit, generated ExecutionId on every run (today only Taxonomy/Essay generate one, and it's discarded).
Add a PlaybookReplayContext<TEvidence,TFinding>-aware overload to SqlStatisticsClassificationRunner — cheap, since it already calls the package's PlaybookExecutor directly, which already supports it.
Extend PlaybookWorkflowGraphExecutor (or add a sibling non-graph path) so Taxonomy/Essay can honor Recall/Replay. Because MAF's WorkflowBuilder graphs are awkward to partially skip, I'd recommend: when replay.Mode != Rerun, bypass the MAF graph entirely and call the Collect/Evaluate/Record tools directly in sequence (mirroring what the package's own PlaybookExecutor does) — same governance recorder calls, same result shape, just no WorkflowBuilder for the replay path. This is a deliberate tradeoff (replay executions don't go "through MAF") that should be called out explicitly in docs/governance/playbook-workflow-types.md.
Persist what's needed to rehydrate a prior execution: extend PlaybookMaterializationEntity (or add a sibling entity) with ExecutionId, ReplayMode, and serialized EvidenceSnapshot/FindingSnapshot JSON columns, plus a new EF Core migration (you mentioned you've already cleared/redeployed the DB this session, so this is a good time for a schema change). Wire the existing-but-unused SavePlaybookMaterializationCommand into each Record tool so every run actually gets persisted.
Governance: since the installed Goodtocode.Agents.Governance v2.0.5 doesn't yet have first-class ReplayMode/SourceExecutionRef fields (confirmed via reflection — only present on the package's unreleased main branch), I'd encode replay info into existing fields as a stopgap (e.g. suffix PolicyProfileVersion, add a GovernanceReference to EvidenceRefs pointing at SourceExecutionId) and flag in docs that upgrading to a future governance package release should replace the stopgap.
The open UX question you raised — my evaluation:
Given this is a quick-start template (bias toward low complexity/high transparency), I'd recommend starting with option (a): simple re-execute, not before/after — specifically:
Each playbook page gets a "Rerun" button (always available) and, once an ExecutionId exists for the currently displayed result, "Recall" and "Replay" buttons.
Clicking Recall or Replay replaces the displayed result in place, but the playbook summary card explicitly shows which mode produced it (Rerun / Recall / Replay + Source Execution Id) — so transparency comes from labeling, not from a side-by-side diff.
A true before/after comparison view is a reasonable phase 2 once the persistence (step 5 above) exists, since at that point comparing two persisted executions is just a query, not a new domain concept — I'd defer it rather than build it alongside the base persistence.
This keeps Plan 2's first slice additive (no destructive UI pattern), avoids inventing a comparison UI before we know users want it, and still fully exercises all three PlaybookReplayMode values end-to-end.
Activity