Skip to content

Documents, exports and versions

How to operate a workflow at volume (T36): the documents table of a workflow (S3), the exports (CSV, Excel, JSON, JSON Lines, Markdown ZIP), and the safe cycle of change in the Studio — test a draft, see the diff, publish with a suggested semantic version, move aliases, restore.

The documents screen

/workflows/<id>/documents shows one row per document (run) and one column per field of the workflow. The state of each row is one of seven, mapped from the run and its review (web/src/features/documents/model.ts, mirrored by exports/states.py):

State Run Review
Not started no run yet
Queued queued
In progress running
Processed succeeded, partially_succeeded, waiting_review not verified
Verified as Processed review_document.state = 'verified'
Error failed
Cancelled cancelled

Listing: GET /v1/runs

Keyset paging on (sort key, id). There is no offset (a 400, RFC 9457) and no total: the response is {data, next_cursor, has_more} and the cursor is opaque and signed, bound to the filters and the sort it was made with (another view's cursor is 400 invalid-cursor).

Parameter Meaning
workflow wf_… or the workflow's name
status a run status (failed, waiting_review…) or one of the table states (processed, verified, error…)
version wfv_…
q find a value in any field: exact (INV-2026-9001) or contained, case-insensitive, over the effective values
confidence_lt runs with a field below this confidence
rule_failed true: with a failed rule, false: without
purpose production (default), test, eval… or all
sort created_at (default), finished_at, status, confidence, cost; - for descending
fields comma-separated field paths whose effective value goes in each row (the visible columns)
cursor, limit paging (limit ≤ 200)

Search and the columns read field_value_index (one row per run and field, with value_norm under a trigram GIN index) and run_doc_index (minimum confidence, failed rules). They are rebuilt on read when a run closed, was approved (runs.event_seq, which extraction.corrected and review.completed bump) or got a reviewer's decision (review_document.last_seq), so a page never shows stale data. Certified deletion and ZDR delete both tables with the run's other rows.

Bulk actions: POST /v1/runs:batch

{"action": "resume", "selection": {"mode": "exclude", "ids": ["run_a", "run_b"],
                                   "filter": {"workflow_id": "…", "status": "failed"}}}

action is resume (retry from the failed step: the cache keeps what finished), rerun (a new run per document, version_id for another version) or cancel. The server resolves the selection: include names runs; exclude means every run the filter matches except ids. "Select all, untick 2" therefore acts on exactly N − 2 runs however many pages the client loaded, and the client never sends a list of thousands of ids. Up to 500 runs are acted on before the answer; a larger selection answers 202 with status: processing and finishes in the background. A run that cannot take the action is skipped with its reason (not_resumable, already_finished…); one failure never stops the others.

Uploading

Each file is one run: POST /v1/runs multipart with workflow (facturas@prod), the file and priority, and an Idempotency-Key derived from the file's SHA-256 and the target. The queue sends at most four files at once; sending the same file again replays the first run (Idempotent-Replayed: true) and creates nothing. A file over 25 MB is refused in the queue with a message. The priority goes to the run queue of T14; concurrency per project is T14's.

Columns

GET/PUT /v1/workflows/{id}/documents/columns keep the columns a user shows, per workflow (export_column_preference); GET also returns available (every scalar field and table the runs have). An export with columns: "visible" uses them.

Exports: POST /v1/exports

{"workflow_id": "wf_…", "format": "csv", "filters": {"status": "waiting_review"},
 "columns": "visible", "layout": "documents", "selection": {"mode": "exclude", "ids": ["run_x"]}}

The request answers 202 at once (also for 10 000 runs): it stores an ExportJob and submits a runtime run of kind export. GET /v1/exports/{id} is pending, running, ready (with a download link) or failed (with its reason: a failure, or an export that matched no run). GET /v1/exports?status=pending is what the notification bell polls every 2 s; the file is served by GET /v1/exports/{id}/download (409 export-not-ready before it exists).

Format Contents
csv a header and a run_id column; no byte-order mark; a null is an empty cell, an empty array []; text starting with = + - @ is prefixed with '
xlsx openpyxl (MIT) in write_only mode: sheet Documents and one sheet per table field; formula-like text is stored as text
json {layout, columns, data: […]}, one element per document with fields, tables and the nested effective data
jsonl one valid JSON object per line
markdown_zip exactly one <run_id>.md per run: header, the effective values and the document's anchored markdown (T04)

layout: documents is one row per run; rows is one per table row. Values are the effective ones: the extraction with the reviewers' corrections, read through T32's review service (exports/effective.py) — never from the sealed tables. A run whose data was erased exports redacted. The writers stream (memory stays flat) and use no dataframe library. "Exportar vista actual (.xlsx)" writes the loaded rows in the browser (hand-written SpreadsheetML zipped with fflate, MIT, a deferred chunk; up to 5 000 rows) without touching the server.

The cycle of change in the Studio

The operate bar of the canvas (/workflows/<id>) has the version selector, the aliases (prod v3.2), Probar, Comparar, Publicar and Documentos.

Test panel

Probar runs the draft on 1–5 documents: one POST /v1/runs per file with purpose: "test" and the draft's {version_id} (only a test run may take a draft). A sixth document is 422 too-many-test-documents (and refused in the panel). The panel shows, per node and document, state, duration, cost and cache, plus the total cost and the latency p50 and p95. Test runs stay out of the default listing, the node cache production reads, the workflow statistics and the datasets.

Diff: GET /v1/workflow-versions/{id}:diff?against={other}

Besides the structural lists of T25 it answers summary, a flat changes[{kind, op, path, before, after, breaking}] and semver_bump. A field with the same field_id and another key is one rename (never an add plus a remove). The Studio draws the same diff on the read-only canvas: green added, red removed, amber modified — each with its text label — next to the list of changes.

Publishing and semantic versions

POST /v1/workflow-versions/{id}:publish {change_summary?, bump?} makes the draft an immutable version (PUT afterwards is 409 version-immutable). The number comes from the diff against the latest published version:

semver_bump When
major the output shape changes: fields added, removed or retyped (also the contract of the output)
minor nodes, edges, rules or structural configuration change
patch only descriptions or notes change
none nothing changed: 409 NOTHING_TO_PUBLISH, and the dialog blocks the button

The author may choose a higher bump; a lower one is 422 bump-too-low (the dialog only offers the allowed ones). The first publication is 1.0.0. With require_evaluation_to_publish on the workflow (PATCH /v1/workflows/{id}, default off) :publish is 409 evaluation-required until the workflow has a completed evaluation; the dialog warns when the previous version has none (GET /v1/evaluations?version_id=&limit=1; any non-2xx or empty answer counts as "no evaluation").

Aliases

PUT /v1/workflows/{id}/aliases/{alias} points dev, staging, prod or a custom name (^[a-z0-9-]{1,32}$, otherwise 422) at a published version. Publishing and moving the alias are two operations that the dialog chains; moving an alias never changes a version. POST /v1/runs with facturas@prod resolves the alias once, runs that version and records the resolved version_id. Restoring (POST /v1/workflows/{id}/versions {from_version_id}) creates a draft with the same graph hash as the restored version — also from an archived one — and touches no published version.

Where things live

What Where
Listing, filters, keyset src/extractit/exports/{filters,listing,indexer}.py, api/routers/runs.py
Bulk actions exports/{batch,selection}.py, api/routers/runs_batch.py
Exports exports/{service,effective,handlers}.py, exports/writers/, api/routers/exports.py
Tables migration 0017_t36: export_job, run_doc_index, field_value_index, export_column_preference, workflows.require_evaluation_to_publish
Screen web/src/features/documents/
Studio operate bar web/src/features/studio/{StudioOps,VersionSelector,AliasManager,PublishDialog,DiffView,DiffCanvas,ChangeList,TestPanel,SamplePicker,NodeResultsTable}.tsx