Skip to content

Monitoring by version (T40)

The Monitoring tab of a workflow's Health screens (/workflows/{id}/monitoring, S12) measures each published version in production: KPI cards, time series with a vertical line at each publication, a version filter, the comparison of two versions, the confidence histogram, the production reliability diagram and the alerts. Every chart has an accessible data table; a metric that cannot be computed reads «No disponible» with its reason, never 0. The definitions are in the monitoring formulas.

API

Route Permission What
GET /v1/workflows/{id}/metrics?version=&from=&to=&granularity=hour\|day&group_by=version workflow:read {series[], confidence_histogram{kind}, markers[], totals}
GET /v1/workflows/{id}/monitoring/quality?from=&to= workflow:read silent error with interval, n_audited, accepted-document error, drift, policy state
GET /v1/workflows/{id}/monitoring/reliability?calibrator_id=&from=&to= workflow:read 15 equal-mass bins, ECE, noise floor, calibrator
GET /v1/workflows/{id}/alerts workflow:read rules and the latest 50 firings
POST /v1/workflows/{id}/alerts workflow:write create a rule (Builder, Admin, Owner)
DELETE /v1/workflows/{id}/alerts/{rule_id} workflow:write delete a rule

Answers are aggregated on demand (indexed (workflow, version, finished_at) over runs) and cached 60 s in process per tenant and parameters. A window is at most 90 days. Another tenant's or project's workflow is 404.

Alerts

Kinds: stp_below, review_rate_above, error_rate_above, p95_latency_above, accuracy_below, silent_error_upper_above. A rule fires once when its metric crosses the threshold (an alert_event row and an alert.fired event in the outbox), does not repeat while it stays there and re-arms when the metric recovers by hysteresis (a fraction of the threshold, 2 % by default). silent_error_upper_above without a threshold uses the largest α of the threshold policy; when a policy exists and the workflow has no such rule, the evaluator creates one.

Events (outbox of T14, delivered by T27 — this ticket sends nothing to Slack, email or webhooks):

Event When
alert.fired a rule crosses its threshold
threshold.breach the silent-error upper bound exceeds α_ref; the policy is tightened once
calibration_drift PSI > 0.2 or rolling ECE > noise floor + 0.01

T14's outbox is per run, so these workflow-level events go to the outbox of the workflow's latest production run (alert_event.run_id, outbox_seq).

The evaluator (AlertService.evaluate) also labels the audit samples whose review was verified (AuditLabeler). quality/monitoring/schedule.py runs it every 5 minutes as a DBOS scheduled workflow (dx_evaluate_monitoring) over the workflows with enabled rules.

Data

Migration 0026_t40 (RLS on all four tables): alert_rule, alert_event, run_confidence (the 20-bin score histogram of each finished run, written when the workflow result is composed) and monitoring_state (once-until-recovery state of the breach and drift guards).