Skip to content

Workflows: the typed graph

A workflow is a typed graph of 14 node types. extractit.workflows defines the graph, checks it (every violation in one call), compiles it to a deterministic plan, stores it as immutable semantic versions and serves it over /v1/workflows. It never runs anything: the interpreter is T26, the Studio canvas is T34.

# doc-test
from extractit.core.schema import load_schema
from extractit.workflows import WorkflowGraph, compile_plan, graph_hash, validate_topology

schema = load_schema(
    {"type": "object", "properties": {"invoice_number": {"type": "string"}}}, id_namespace="doc"
)
graph = WorkflowGraph.model_validate(
    {
        "nodes": [
            {"id": "p", "type": "parse"},
            {"id": "e", "type": "extract", "config": {"schema_id": "invoice"}},
        ],
        "edges": [{"source": "p", "target": "e"}],
    }
)
report = validate_topology(graph, {"invoice": schema})
assert report.valid
plan = compile_plan(graph, {"invoice": schema})
assert [s.node_id for s in plan.steps] == ["p", "e"]
assert len(graph_hash(graph, {"invoice": schema})) == 64

The JSON Schema of the graph is schemas/workflow-graph.schema.json (contract workflow-graph 1.1: T49 added the model_policy fragment of extract, see Model policies); regenerate it with python -m extractit.workflows.graph --out schemas/workflow-graph.schema.json (a contract test and make contracts fail when it is outdated).

Node catalog

Every node has id ([a-z0-9_]{1,64}), type, schema_version (semver of the node's own contract), optional name and optional canvas position ui. Its config is closed: unknown keys are rejected, and an unknown type is rejected with the list of the 14 valid ones.

Type Studio label Configuration
parse Parse profile
extract Extract schema_id, tier (fast/standard/agentic), model_policy, lookups, prompt_addendum
classify Classify categories (tree, up to 3 levels), instructions, threshold, multi_label, max_labels, unclassified_branch
split Split rules[{id, name, description, partition_key}], allow_non_contiguous, unassigned_branch, min_confidence
validate Validate rules (T09 ValidationRule)
branch If/else condition (T09 Check)
review Review policy{when, threshold, queue, timeout_hours}
transform Code language, code, output_schema, limits
fill_form Edit instructions, reference_ids, output_mode, font, reference_table (Edit node)
report Report template, format (Report node)
knowledge Knowledge index
notify Notify channel, target, template, title, severity, recipients
export Export destination, path, format, mapping, rows_from, on_failure
plugin Plugin plugin_id, version, config, ports{inputs, outputs}

Edges are {source, target, branch?}. branch selects an output of a branching node: true or false for branch, a category id or unclassified for classify, a rule id or unassigned for split. annotations are opaque sticky notes of the canvas.

Ports

Five port types travel along the edges: ParsedDoc, Segment, Extraction, Verdicts, File. An edge s -> t is valid when accepts(t) intersects available(s), where available(s) = emits(s) ∪ (carries(s) ∩ ⋃ available(predecessors of s)).

Node Accepts Emits Carries
parse — ParsedDoc —
classify ParsedDoc ParsedDoc everything
split ParsedDoc Segment ParsedDoc
extract ParsedDoc, Segment Extraction ParsedDoc, Segment
validate Extraction Verdicts Extraction, ParsedDoc, Segment
branch Extraction, Verdicts — everything
review Extraction, Verdicts Extraction everything
transform, fill_form ParsedDoc, Extraction File — (terminal)
report Extraction File — (terminal)
knowledge ParsedDoc, Extraction, Verdicts — — (terminal)
notify Extraction, Verdicts — — (terminal)
export Extraction, Verdicts, File — — (terminal)
plugin its ports.inputs its ports.outputs —

So parse → extract → validate → branch → extract is valid (WF-07, Anyformat rejects it) and parse → extract → notify → extract is a port_mismatch: a terminal forwards nothing. This is a deliberate divergence from Anyformat, covered by tests.

Topology rules

validate_topology(graph, schemas, ctx=None) is pure and never stops at the first violation. It returns errors and warnings, each with a stable rule, a message, the node_ids involved and a JSON Pointer relative to the graph (/nodes/3, /edges/0), sorted by (rule, node_ids, pointer, message). Publishing a graph with errors is a 400 TOPOLOGY_INVALID with the same list in errors[].

Rule Fires when
parse_count there is not exactly one parse node (the empty graph is a valid draft, not publishable)
duplicate_node_id two nodes share an id
orphan_node a node other than parse has no incoming edge
cycle nodes form a cycle (one violation per strongly connected component)
invalid_fanout a node other than classify, split, branch has more than one successor
single_input validate, branch or notify has more than one predecessor
split_rule_shared_target two rules of a split lead to the same node
duplicate_category_or_rule repeated category ids (whole tree), split rule ids or validate rule ids
duplicate_branch a branch/classify/split has two edges with the same label
branch_missing_true a branch has no true edge
extract_requires_field the schema of an extract has no fields
result_key_collision two sibling fields of a schema share a snake_case key
multiple_knowledge more than one knowledge node
rule_ref_not_upstream a validate rule or branch condition reads a field no extract upstream defines
terminal_has_outgoing a terminal node has outgoing edges
port_mismatch the target accepts nothing the source makes available
edge_unknown_node an edge points to a node that does not exist (node_ids lists the existing end)
invalid_branch_label the label is not valid for the source (also an unlabeled branch edge)
unknown_schema schema_id is not in the version's schemas
plugin_ports_undeclared a plugin declares no port (config or manifest)
reserved_branch_id a category or rule is called true, false, unclassified or unassigned
empty_false_branch (warning) a branch has no false edge
unreachable_branch (warning) an edge uses unclassified/unassigned while the node never emits it

The first 15 are the rules of parity with Anyformat; the next six are ours. Extension rules registered with register_topology_rule(code, fn) (T28 classify/split, T30 plugins, T27/T31 templates) run after them (T28 adds classify_categories_invalid, classify_unclassified_edge_unknown, classify_unclassified_missing and classify_many_leaves: see Classify and Split), and register_node_config(type, model, schema_version=…) tightens a node's configuration: the validator then also reports invalid_node_config and unsupported_node_version. extract_requires_field and result_key_collision guard schemas that did not come through load_schema (T02 already refuses them at load, so over HTTP they surface as a 400 INVALID_SCHEMAS).

Compiler and hashes

compile_plan(graph, schemas) validates and returns an ExecutionPlan: a stable topological order (ties by node id) of PlanStep{node_id, node_type, resource_class, depends_on, scope_expansion, hitl_point, cacheable, config_hash}. resource_class maps to the queues of T14: parse and transform → cpu; classify, split, extract and validate with AI rules → llm; notify, export, report, knowledge (and validate with external rules) → io; branch, review and deterministic validate → inline. split expands the scope per_segment, classify with labeled edges per_category. plan_hash is the sha256 of the JCS (RFC 8785) bytes of the plan.

graph_hash is the sha256 of the JCS bytes of {graph, schemas} without annotations and ui, with nodes sorted by id and edges by (source, target, branch). Order, whitespace, notes and node positions never change it; any configuration change does. derive_output_contract returns a JSON Schema with extractions.<node_id>, classifications, segments and artifacts.

Versions

A workflow has at most one draft (editable, saved even with violations), any number of published versions (immutable) and archived ones. Publishing copies the extraction schemas inside the version, computes graph_hash and the semantic version against the highest published or archived version. Send back the normalized schemas of a response (they carry the x-dx.field_id that survive renames); fields without one get a deterministic id.

Change from the previous version Level Example
first publication — 1.0.0
identical graph_hash none 409 NOTHING_TO_PUBLISH
only descriptive text (description, instructions, prompt_addendum, name, title, enum_descriptions) patch 1.0.0 → 1.0.1
edges, node types or configuration (tier, rules, thresholds, schema_id, templates…) minor add a notify node: 1.0.0 → 1.1.0
the output contract: field added, removed, renamed, retyped or required; labels, segments or artifacts changed major rename an output key: 1.0.0 → 2.0.0

The highest level detected wins. tests/fixtures/workflows/semver_pairs.json holds 30 before/after pairs. Immutability has two layers: the repository exposes no UPDATE of published rows, and a Postgres trigger rejects any change of graph, schemas, graph_hash, number, parent_version_id or output_contract on a non-draft row (only published → archived is allowed). Restore (POST …/versions {from_version_id}) creates the draft from any version.

Aliases

An alias (dev, staging, prod, or any [a-z][a-z0-9-]{0,31}) points at a published version; ALIAS_TARGET_NOT_PUBLISHED (422) for a draft or archived one. Every move is recorded in the alias history with author and date. A version an alias points at cannot be archived. WorkflowService.resolve(ref) accepts name@alias, name@3.2.1 and wfv_<id>; a bare name is ALIAS_REQUIRED.

REST API

Conventions of the platform apply (see API conventions): Idempotency-Key on every POST, ETag/If-Match on drafts and workflows, keyset pages, one audit event per mutation, workflow:read|write|publish permissions. Operations tagged mcp are exposed as MCP tools.

Operation Permission
POST/GET /v1/workflows, GET/PATCH/DELETE /v1/workflows/{id} (DELETE archives) write / read
POST /v1/workflows/{id}:validate (dry run, 200 with the report) read
GET/POST /v1/workflows/{id}/versions (POST creates the draft or restores) read / write
GET/PUT /v1/workflow-versions/{id} (PUT only a draft, If-Match = revision) read / write
POST /v1/workflow-versions/{id}:publish, :archive publish
GET /v1/workflow-versions/{a}:diff?against={b} read
GET /v1/workflows/{id}/aliases, PUT/DELETE …/aliases/{alias}, GET …/aliases/{alias}/history read / publish

Error codes: TOPOLOGY_INVALID (400), INVALID_SCHEMAS (400), NOTHING_TO_PUBLISH, version-immutable, DRAFT_EXISTS, VERSION_IN_USE, WORKFLOW_NAME_TAKEN, WORKFLOW_ARCHIVED (409), ALIAS_TARGET_NOT_PUBLISHED, ALIAS_REQUIRED, INVALID_REFERENCE (422), plus the platform ones (precondition-failed carries the current ETag).

diff describes what changed going from against to the version: nodes_added, nodes_removed, nodes_changed[{path, before, after}], edges_added, edges_removed, fields_added, fields_removed, fields_renamed (same field_id, another key), rules_changed and severity.

WorkflowService.on_published(hook) registers hook(session, event), run inside the publishing transaction: T27 emits workflow.version.published from there.