Review inbox (S8) and human review inside a run
A workflow with a review node pauses the run when a document needs a person, hands the work out
through an inbox (/review) and wakes the run up again when the last task is done — without
repeating any step. This page covers the policy of the node, the tasks, assignment, the SLA, double
review, the permission matrix and the contract with the interpreter (T26). The review screen itself
(fields, decisions, verification) is Review in the viewer.
The review node policy
ReviewPolicy is part of the node configuration; its JSON Schema is
schemas/review/review_policy.schema.json (and the node-type registry serves it to the node panel of
the canvas). Every field is optional.
| Field | Meaning | Default |
|---|---|---|
when |
always, on_blocking_failure, on_low_confidence, on_unclassified, on_sampling |
on_blocking_failure |
threshold |
confidence below which a field (or the document) is reviewed (on_low_confidence) |
0.8 |
sampling_rate |
share of runs the QA sample takes (on_sampling), 0–1 |
0 |
auto_approve.threshold_policy_id |
thp_…: a certified threshold policy of calibrated confidence decides instead of when |
none |
queue |
queue the tasks go to | default |
assignment |
manual or round_robin |
manual |
reviewers |
usr_… ids of the round-robin; empty: the members with Reviewer role or above |
[] |
sla_minutes |
minutes from creation to due_at |
1440 |
escalate_to |
usr_… that receives a task when its SLA expires |
none |
double_review |
two reviewers work in isolation, a third adjudicates disagreements | false |
audit_sla_minutes |
SLA of the random-audit tasks | 2880 |
timeout_hours |
how long the run waits before the node fails with REVIEW_TIMEOUT |
168 |
Canonical names and aliases. The stored (and hashed) spelling is the one T26 introduced: when
and threshold. The ticket's spellings are accepted on input and normalized to the canonical ones,
so a workflow written either way has the same graph hash:
| You may write | Stored as |
|---|---|
trigger: always |
when: always |
trigger: low_confidence |
when: on_low_confidence |
trigger: rule_failure |
when: on_blocking_failure |
trigger: sampling |
when: on_sampling |
confidence_below |
threshold |
Which documents need a person
when |
Opens a task when… | Reason recorded |
|---|---|---|
always |
always | always |
on_blocking_failure |
a blocking rule failed upstream | blocking_failure + the failed rule ids |
on_low_confidence |
a field, or the document, has confidence < threshold |
low_confidence, and fields_to_review lists those fields |
on_unclassified |
a classification left the document unclassified | unclassified |
on_sampling |
the QA sample takes the run | qa_sampling |
The QA sample is deterministic: sha256(workflow_id:run_id) mod 10 000 < sampling_rate × 10 000, so
the same run always gets the same decision and the rate holds over a population (0.1 over 1 000 fixed
runs gives 100 ± 15 tasks). It is a quality check with a public seed; the secret audit sample of
T39 is separate (below). When there is nothing to review (no field below the threshold, every field
pre-approved) no task is created and the node continues.
Auto-approval and random audit (CNF-04, CNF-05)
With auto_approve.threshold_policy_id the node asks ThresholdService.decide(result, validations,
policy_id) of T39:
- auto-approved (every required field above its λ and no blocking rule failed): no task, the run
continues. The document is then offered to
AuditSampler.consider; when the sample takes it a task of kindauditenters the queueauditwith reasonrandom_auditanddue_at = created_at + audit_sla_minutes. The run does not wait for it; verifying it in the review screen is what labels the sample for T40. It escalates with the same sweep as any task. - not auto-approved: a normal task whose reasons are the decision's (
required_below_threshold,blocking_rule_failed, …) and whosefields_to_revieware the fields that did not clear their λ. - no decision possible (no such policy, an output that does not parse): the safe answer is a
person, with reason
auto_approval_unavailable. Nothing is ever auto-approved without a valid policy.
Tasks
review_task (rt_…) is one unit of work: a run, a node, a role and a due date.
| Field | |
|---|---|
state |
open → in_progress (claimed) → completed; escalated after the SLA; expired when the run ended without it |
kind |
review (the run waits for it) or audit (it does not) |
queue, reason[], fields_to_review[] |
where it sits, why, and what to look at |
assignee |
usr_…; with an API key the actor is the key (key_…), not a person |
task_role, group_id |
primary, secondary (double review) or adjudicator |
due_at, priority |
the keyset listing orders by (due_at, id): most urgent first |
A run is idempotent per (run, node, scope, kind, role): opening the same task twice returns the
first one. Cancelling a run — or one that fails — moves its pending tasks to expired; completing one
of them then answers 409.
API
| Route | Needs | What it does |
|---|---|---|
GET /v1/review-tasks?queue=&state=&assignee=&reason=&workflow_id=&kind=&cursor=&limit= |
result:read |
the inbox (state=pending is open + in progress + escalated; assignee=me\|unassigned\|usr_…) |
GET /v1/review-tasks/summary |
result:read |
counters: open (the badge of the sidebar), waiting, in_progress, escalated, overdue, mine, unassigned, by_queue |
GET /v1/review-tasks/{id} |
result:read |
one task; an adjudicator task also carries both reviewers' values |
GET /v1/review-tasks/{id}/decisions |
result:read |
the decisions of that task (a reviewer of a double review asking for the other's gets 404) |
POST /v1/review-tasks/{id}:claim / :release |
review:act + assignee |
take / give back a task |
POST /v1/review-tasks/{id}:complete |
review:act + assignee |
verify (or reject) the document for this task; idempotent |
POST /v1/review-tasks:assign |
project:manage |
{task_ids \| filter + exclude, assignee \| strategy: round_robin, reviewers?}; up to 1 000 per call |
GET /v1/review-reviewers |
project:manage |
who a task can be assigned to |
GET /v1/review-metrics?since=&until=&reviewer= |
result:read |
per reviewer: documents/hour, % of corrected fields, double-review agreement |
Errors are RFC 9457 problems: not-assigned-verifier (403), insufficient-scope (403, an API key
without the review scope), conflict (409), not-found (404). Everything of T32 (PATCH /fields,
:approve, :reject, rows) goes through the same guard.
Assignment
- Manual:
assigneeis set on every task of the call. - Round-robin: a counter per
(tenant, project, queue)moves with oneINSERT … ON CONFLICT DO UPDATE … RETURNING, so concurrent batches never read the same position and a batch overkreviewers gives each⌊n/k⌋or⌈n/k⌉tasks (difference at most one, whatever the counter was). The next batch continues where the last stopped. Without reviewers nobody is assigned: the task waits in the queue. - Closed tasks in a selection are skipped and reported in
skipped. 500 tasks assign in under 2 s. - An Admin can reassign a task in progress; the previous assignee's claim is cleared.
SLA and escalation
due_at = created_at + sla_minutes (audit tasks: audit_sla_minutes). The counter of the inbox is
green, amber in the last fifth of the SLA and red from due_at on; the colour is never the
only signal (the state is written next to it). The counter refreshes every second in the browser; the
list refetches every 10 s.
A scheduled DBOS workflow, dx_review_sweep_sla (every 60 s, idempotent), does three things:
- moves open or in-progress tasks past
due_attoescalatedand assigns them toescalate_to(a task that was already escalated is not touched, so running the sweep twice changes nothing); - expires the overdue tasks of runs that were cancelled or failed;
- redelivers the wake-up signal of runs that are still
waiting_reviewalthough every task is done (a crash between the verdict and the signal).
Double review
With double_review: true the node opens two independent tasks (primary, secondary) with the same
group_id. Each reviewer works in isolation: their decisions are stored with the task and
applied = false, so they do not change the effective value, and GET …/decisions of the other task
is 404. One person cannot complete both (409).
When both are done the decisions are compared field by field:
- agree → the agreement is applied as
system:double_review, the document is verified (onereview.completed) and the run resumes; - disagree (in several fields at once is fine) → what they agree on is applied now and an
adjudicatortask opens with both reviewers' values for each disputed field (assigned toescalate_towhen set). Its decision is applied as an override and verifies the document.
The agreement metric is matching fields / compared fields; a field only one reviewer touched is
compared against the original value.
Permission matrix
Viewer / read key |
Reviewer | Admin / Owner | |
|---|---|---|---|
| List tasks, summary, metrics, decisions of a task | yes | yes | yes |
| Claim, release, complete | 403 | their own tasks | any task |
PATCH /fields, rows, :approve, :reject on a run with a pending task |
403 | only if assigned (else 403 not-assigned-verifier) |
yes |
| Same on a run without a pending task | 403 | yes | yes |
| Assign in batch, reassign, list reviewers | 403 | 403 | yes |
A task nobody holds must be claimed first (claiming assigns it); completing one assigns it to the
caller. Scopes are checked before roles: an API key without the review scope gets
insufficient-scope. With an API key the actor of a task is the key, so metrics and completed_by
name key_…, not a person. Another tenant's task or run is always 404.
The screen (S8)
/review (and /workflows/{id}/review, which filters by workflow):
- the table is windowed with
@tanstack/react-virtualover@tanstack/react-table: a few dozen rows are in the DOM whatever the size of the queue (at most 60), more pages load as you scroll; - filters (queue, reason, assignee, state) live in the URL;
- assign: tick tasks (or «select every task that matches the filters» and untick the exceptions), then choose a reviewer or round-robin (Admin only);
- focus mode (button, or the Review action of a row) opens the first task in the review screen;
the review's «verify and next» (Shift+Enter) verifies the document and opens the next task in
queue order, and the last one ends with «Cola vacía». A task nobody holds is claimed before it
opens.
Shift+←goes back; - the metrics tab shows each reviewer's figures with the counts they come from;
- the badge next to «Revisión» in the sidebar is
summary.open, refreshed every 10 s.
Contract with the interpreter (T26)
The node is T26's review handler, extended — not duplicated. GateReviewTasks
(src/extractit/review/gate.py, the only place coupled to the interpreter) implements T26's
ReviewTaskPort.
- Open. The handler evaluates the policy over the extraction, validations and classification of
the run. With a reason it marks the node
waiting, callsports.review.open(...)(which creates the task(s) idempotently, in the same transaction) and emitsnode.waiting. - Pause. It parks on T14's
ctx.wait_for(review_topic(node_id, scope_key), timeout): the run iswaiting_review(eventrun.waiting_review, one per run), leaves its queue slot and holds no thread, connection or worker. It survives restarts andkill -9: the task and itsdue_atare rows, the wait is a durable DBOS parking. - Signal. Completing the last task verifies the document with T32's
ReviewService(exactly onereview.completed) and, after the verdict is committed, sends the payload{"decision": "approve"|"reject", "corrections": {...}, "effective_data": {...}, "reviewer": "…"}tocomplete_reviewwith the idempotency keysha256(run_id:node_id:review). A second delivery answersnoop. Verifying the document directly (POST /v1/runs/{id}:approve) closes its pending tasks and wakes the node the same way.POST /v1/runs/{id}/nodes/{node_id}:completeremains the low-level door. - Resume. The node leaves
waiting, applies the reviewer's values to the extraction (review_apply) and continues;rejectfails the node withREVIEW_REJECTED; the timeout fails it withREVIEW_TIMEOUTand expires its tasks. Parse and extract are not repeated: their node runs aresucceededwith one attempt, and the replay of the interpreter reuses them. - Cancel. Cancelling a
waiting_reviewrun finalizes it and expires the tasks it still had.
Until the run finishes, GET /v1/runs/{id}/result of a run waiting for its review answers a
provisional result (the extraction as it stands), so the review screen can open it.
Tests: tests/review_tasks/ (policy, assignment, SLA, double review, guard, metrics, listing at
scale) and tests/workflows/interpreter/test_review_tasks_run*.py (the real interpreter: pause,
kill -9, resume, double review, cancellation, auto-approval, audit tasks);
web/src/features/inbox/ and web/e2e/mocked/inbox.spec.ts for the screen.