Validation rules
extractit.rules evaluates rules on an extraction and returns one verdict per rule. A single
engine covers deterministic checks, CEL expressions, AI rules written in natural language and external
validators, so the pipeline, workflows, derived fields and the editor all behave the same. The engine
is pure and reproducible for deterministic rules: same rules and same extraction, same results.
# doc-test
from extractit.core.ports import ExtractionView
from extractit.core.results import ExtractedField
from extractit.core.schema import load_schema
from extractit.rules import evaluate, has_blocking_failure, load_rules
schema = load_schema(
{
"type": "object",
"properties": {
"net": {"type": "number"},
"tax": {"type": "number"},
"total": {"type": "number"},
},
}
)
rules = load_rules(
[
{
"id": "totals-add-up",
"kind": "deterministic",
"blocking": True,
"check": {
"type": "arithmetic",
"operands": ["net", "tax"],
"operator": "sum",
"equals": "total",
},
}
],
schema,
) # names are rewritten to field_id: renaming a field later does not break the rule
ids = {f.name: f.field_id for f in schema.fields()}
view = ExtractionView(
[
ExtractedField(path="net", field_id=ids["net"], value=4176.93, status="extracted"),
ExtractedField(path="tax", field_id=ids["tax"], value=417.69, status="extracted"),
ExtractedField(path="total", field_id=ids["total"], value=4600.0, status="extracted"),
]
)
(result,) = evaluate(rules, view)
print(result.status, "-", result.detail)
# fail - 4176,93 + 417,69 = 4594,62 ≠ 4600 (tolerancia 0,01)
assert has_blocking_failure([result])
Details are written in Spanish, the language of the first documents the product processes.
Statuses
| Status | Meaning |
|---|---|
pass |
The rule holds. |
fail |
The rule does not hold. |
inconclusive |
It cannot be decided; inconclusive_reason says why (below). It is never a pass. |
skipped |
applies_when is false: the rule does not apply to this document. |
inconclusive_reason: missing_field (the field is absent or has no value), type_error (the value
cannot be converted, e.g. text where a number is needed), expression_error (a CEL runtime error or an
internal error), model_unavailable (the AI judge could not answer), connector_unavailable,
connector_timeout and connector_rate_limited (external validators) and no_docir_table
(row_count against a DocIR table that does not exist). Any failure while evaluating a rule becomes an
inconclusive verdict, never an exception.
Combinators use three-valued (Kleene) logic: all_of fails if any child fails, else it is
inconclusive if any child is, else it passes; any_of passes if any child passes, else it is
inconclusive if any child is, else it fails; not swaps pass and fail and keeps inconclusive.
A rule
{
"id": "totals-add-up",
"name": "Net plus tax equals the total",
"kind": "deterministic",
"severity": "error",
"blocking": true,
"applies_when": {"type": "comparison", "left": "currency", "op": "==", "right": {"literal": "EUR"}},
"check": {"type": "arithmetic", "operands": ["net_total", "tax_total"], "operator": "sum", "equals": "gross_total"},
"version": 1,
"on_inconclusive": "review"
}
| Field | Notes |
|---|---|
kind |
deterministic (uses check), ai (uses prompt and source_field_ids) or external (uses external). |
severity |
error, warning or info; copied to the result. |
blocking |
When true, a fail makes has_blocking_failure(results) true (review is mandatory before exporting). |
on_inconclusive |
review (default) or ignore. A blocking rule that is inconclusive requires review (requires_review(results, rules)) unless it says ignore. |
applies_when |
Any check. False → skipped; inconclusive → inconclusive; true → the rule is evaluated. |
source_field_ids |
The fields a rule is about: their evidence is attached to the verdict (and, for AI rules, they are the only data the judge sees). |
The JSON Schema of a rule is schemas/rules-1.0.json (checked by make contracts); see
examples/rules/ for complete rule sets.
Checks
A field reference is a field_id, a path (supplier.vat_id), a row path (line_items[*].amount) or
a field name. load_rules stores field_id (a concrete row such as line_items[2].amount stays a path).
A [*] reference is evaluated per row: the check fails if any row fails and is inconclusive if
some row is and none fails.
| Type | Parameters | Passes when |
|---|---|---|
required |
field |
the field is present, not not_found/abstained, not empty. |
range |
field, min?, max? |
min <= value <= max (inclusive). Numbers may be tolerant strings (1.234,56, € 12,50, 12 %). |
date |
field, earliest?, latest? |
the ISO date is inside the bounds; a bound is an ISO date or today (injectable clock). |
arithmetic |
operands, operator (sum, subtract, product), equals, tolerance=0.01 |
the operation equals equals within an absolute tolerance. A [*] operand counts as the sum of its rows. Detail: 4176,93 + 417,69 = 4594,62 (tolerancia 0,01). |
comparison |
left, op (==, !=, >, >=, <, <=), right ({"field": …} or {"literal": …}) |
the comparison holds. Numbers, ISO dates and text compare as such; booleans only with ==/!=. |
one_of |
field, allowed, case_sensitive=true |
the value (every element of a multi-select) is in allowed. |
regex |
field, pattern |
the RE2 pattern matches (partial match, like JSON Schema: anchor with ^…$). RE2 runs in linear time, so a pattern such as (a+)+$ cannot hang the engine; look-around and back-references are not supported. |
checksum |
field, algorithm (iban, luhn, nif_es, vat_eu, isbn, lei) |
the check digit is valid. nif_es covers DNI, NIE and CIF; vat_eu is the EU VAT number with country prefix. |
confidence |
field, op=">=", threshold (0–1) |
the field confidence satisfies the comparison. |
evidence |
field, at_least (verified or partial) |
the best non-contradicting evidence is at least that verified. |
row_count |
field, against (declared_field, docir_table or table_stats), count_field?, logical_table_id?, table? |
the extracted rows equal the count printed in the document (count_field), or the rows of the DocIR logical table (without that table: inconclusive / no_docir_table), or — table_stats, declared implicitly for x-dx.table.completeness = all_rows (T22) — the rows the table extraction accounts for (rows_physical - rows_excluded - rows_merged of table_stats[table]): fail with faltan N filas when some are missing. |
expression |
cel |
the CEL expression is true (see below). |
all_of, any_of, not |
checks / check |
Kleene combination of the children. |
required is the only check that fails on a missing field; every other check is inconclusive /
missing_field when the field is absent, not_found or abstained, and inconclusive / type_error
when the value cannot be converted.
CEL expressions
expression checks use CEL (cel-python) over a variable data keyed by
field_id. Tables are lists of objects whose keys are the field_id of their columns; the macros
map, filter, exists and all come with the interpreter. The engine adds three functions:
num(x)converts numbers and strings (1.234,56,1,234.56,€ 12,50,-12.5,12 %) to a double and raises a typed error fornull, booleans and non-numeric text.sum(list)adds withnum(an empty list is0).abs(x)appliesnumand takes the absolute value.
abs(num(data.opening_balance)
+ sum(data.movements.map(m, num(m.credit)))
- sum(data.movements.map(m, num(m.debit)))
- num(data.closing_balance)) <= 0.01
Write expressions with field names: load_rules rewrites them to field_id (string literals are never
touched), and canonicalize_expression(cel, schema, to="names") turns a stored expression back into
names for an editor. An unknown field is a load error with a suggestion (unknown_field_ref).
Runtime errors: a null reaching num is missing_field; a missing key, a division by zero (also for
doubles) or any other error is expression_error; a result that is not a boolean is type_error.
Limits: 4 KiB of text, 32 levels of nesting and 10 000 rows per table. Compilation is cached by canonical
text. A simple precompiled expression runs at several thousand evaluations per second per core
(pytest -k bench_cel).
AI rules
{
"id": "supplier-is-a-company",
"kind": "ai",
"prompt": "The supplier must be a legal entity (a company), not an individual person.",
"source_field_ids": ["supplier_name"]
}
The judge never sees the document: only the values and short citations of source_field_ids, inside a
block delimited by a token derived from the content (a value cannot close the block) with a system
prompt that tells the model to treat it as data. The answer is {status, detail, cited_fields} with
purpose="ai_rule" and temperature 0. cited_fields is filtered to the fields that were provided (the
rest are dropped and logged), the call id goes to model_call_id, and answers are cached by hash of
(rule text, values) for the lifetime of the gateway. If the model is missing or fails the verdict is
inconclusive / model_unavailable. estimate_ai_rule_cost(rules, n_docs) estimates calls and tokens
for an editor.
from extractit.llm.registry import resolve_gateway
results = evaluate(rules, view, gateway=resolve_gateway()) # DX_MODEL=codex, gemini, fake…
External validators
A rule with kind = "external" calls a validator registered by name (VIES, GLEIF, a bank registry…).
The engine substitutes ${field_id} in params with the normalized value of the field (a missing
field is inconclusive / missing_field) and translates the answer:
ExternalResult.status |
Verdict |
|---|---|
ok |
pass |
invalid, not_found |
fail |
unavailable, timeout, rate_limited |
inconclusive with connector_unavailable, connector_timeout, connector_rate_limited |
| validator not registered, or it raised | inconclusive / connector_unavailable |
The network call, caching, rate limits and SSRF protection belong to each validator. A minimal one:
# doc-test
import asyncio
from extractit.core.ports import ExtractionView
from extractit.core.results import ExtractedField
from extractit.rules import (
ExternalResult,
ExternalValidatorRegistry,
ValidationRule,
evaluate,
)
class AlwaysActive:
name = "demo"
async def call(self, operation, params, ctx):
await asyncio.sleep(0) # a real validator awaits its HTTP client here
return ExternalResult(status="ok", source_ref="demo:" + params["vat_number"])
registry = ExternalValidatorRegistry()
registry.register(AlwaysActive())
rule = ValidationRule.model_validate(
{
"id": "vat-active",
"kind": "external",
"external": {"validator": "demo", "operation": "check", "params": {"vat_number": "${vat}"}},
}
)
view = ExtractionView(
[ExtractedField(path="vat", field_id="vat", value="ESA58818501", status="extracted")]
)
(result,) = evaluate([rule], view, registry=registry)
assert result.status == "pass"
Packages register validators under the entry point group extractit.external_validators (a class
with a no-argument constructor, or an instance); ExternalValidatorRegistry(entry_points=True) loads them.
Evidence
Every result has compared_values (what was compared) and evidence_ids: the sorted, deduplicated
union of the evidence of the rule's source_field_ids, the fields its check (or applies_when) reads —
for CEL, every field the expression names — and the fields an AI judge cited. At most 200 ids are kept;
when some are dropped detail ends with (+N evidencias).
API
evaluate(rules, view, *, gateway=None, registry=None, clock=None, docir=None, cache=None)andevaluate_async(...): oneValidationResultper rule, in order. Deterministic rules never start an event loop;evaluatealso works inside a running loop.docir(aParsedDocument) feedsrow_countagainst DocIR tables.RulesValidator(gateway=…, registry=…, clock=…, docir=…)implementsValidatorPort(evaluate(rules, view)); it is registered as the entry pointrulesof the groupextractit.validators.load_rules(path_or_list, schema)resolves references and compiles CEL; it raisesRuleLoadErrorwith every issue (unknown_field_ref,ambiguous_field_ref,field_ref_not_leaf,syntax_error…) and a suggestion when there is one.has_blocking_failure(results)andrequires_review(results, rules).