Summary
Five audit-log processor alerts shipped by this repo page at critical severity in Datum production with no runbook attached. A responder woken by one gets an alert name and a query expression, then has to reconstruct the diagnosis and the recovery under time pressure. That matters more here than elsewhere, because every one of these fires when audit records are being dropped or never arrive. Datum's alerting standard requires a runbook link on every rule, and the procedure for a component's own alert belongs with the component rather than in the deployment repo.
The alerts
Read from production alert-rule compliance metrics on 2026-08-10. All five sit in the vector-audit-log-processor group.
| Alert |
VectorBufferEventsDiscarded |
VectorComponentErrors |
VectorComponentEventsDiscarded |
VectorHttpServerError |
VectorRemapTransformError |
Each one means audit data is at risk: a full buffer discarding events, a transform failing, or the webhook endpoint erroring back at the API server. A responder needs to know how to tell a transient blip from real audit loss, and what to do about the loss once it has happened.
Acceptance criteria
Related to https://github.com/datum-cloud/infra/issues/3727
Summary
Five audit-log processor alerts shipped by this repo page at critical severity in Datum production with no runbook attached. A responder woken by one gets an alert name and a query expression, then has to reconstruct the diagnosis and the recovery under time pressure. That matters more here than elsewhere, because every one of these fires when audit records are being dropped or never arrive. Datum's alerting standard requires a runbook link on every rule, and the procedure for a component's own alert belongs with the component rather than in the deployment repo.
The alerts
Read from production alert-rule compliance metrics on 2026-08-10. All five sit in the
vector-audit-log-processorgroup.VectorBufferEventsDiscardedVectorComponentErrorsVectorComponentEventsDiscardedVectorHttpServerErrorVectorRemapTransformErrorEach one means audit data is at risk: a full buffer discarding events, a transform failing, or the webhook endpoint erroring back at the API server. A responder needs to know how to tell a transient blip from real audit loss, and what to do about the loss once it has happened.
Acceptance criteria
Related to https://github.com/datum-cloud/infra/issues/3727