What & why
Refs #36
Define how ProvenanceBench scores detect.py output and how that maps to aibom_verifier verdicts (verified_derivative, fraudulent_claim, incompatible, unsupported, insufficient_evidence).
Scope
- Document gold
y vs detector pred (related / unrelated / abstain) and confidence float role.
- Threshold policy: when does a float become confirm vs reject vs abstain?
- Map bench labels to verifier taxonomy (table in issue or wiki).
- Optional: per-signal floats in run log for triage; calibration metrics later (not v1 gate).
label_confidence: disputed rows excluded from primary bal_acc or scored separately.
Out of scope
- Implementing real weight signals in
detect.py (separate work).
- Per-run random negative generation.
Test plan
What & why
Refs #36
Define how ProvenanceBench scores
detect.pyoutput and how that maps toaibom_verifierverdicts (verified_derivative,fraudulent_claim,incompatible,unsupported,insufficient_evidence).Scope
yvs detectorpred(related/unrelated/abstain) andconfidencefloat role.label_confidence: disputedrows excluded from primarybal_accor scored separately.Out of scope
detect.py(separate work).Test plan