Skip to content

Validation rules

extractit.rules evaluates rules on an extraction and returns one verdict per rule. A single engine covers deterministic checks, CEL expressions, AI rules written in natural language and external validators, so the pipeline, workflows, derived fields and the editor all behave the same. The engine is pure and reproducible for deterministic rules: same rules and same extraction, same results.

# doc-test
from extractit.core.ports import ExtractionView
from extractit.core.results import ExtractedField
from extractit.core.schema import load_schema
from extractit.rules import evaluate, has_blocking_failure, load_rules

schema = load_schema(
    {
        "type": "object",
        "properties": {
            "net": {"type": "number"},
            "tax": {"type": "number"},
            "total": {"type": "number"},
        },
    }
)
rules = load_rules(
    [
        {
            "id": "totals-add-up",
            "kind": "deterministic",
            "blocking": True,
            "check": {
                "type": "arithmetic",
                "operands": ["net", "tax"],
                "operator": "sum",
                "equals": "total",
            },
        }
    ],
    schema,
)  # names are rewritten to field_id: renaming a field later does not break the rule

ids = {f.name: f.field_id for f in schema.fields()}
view = ExtractionView(
    [
        ExtractedField(path="net", field_id=ids["net"], value=4176.93, status="extracted"),
        ExtractedField(path="tax", field_id=ids["tax"], value=417.69, status="extracted"),
        ExtractedField(path="total", field_id=ids["total"], value=4600.0, status="extracted"),
    ]
)
(result,) = evaluate(rules, view)
print(result.status, "-", result.detail)
# fail - 4176,93 + 417,69 = 4594,62 ≠ 4600 (tolerancia 0,01)
assert has_blocking_failure([result])

Details are written in Spanish, the language of the first documents the product processes.

Statuses

Status Meaning
pass The rule holds.
fail The rule does not hold.
inconclusive It cannot be decided; inconclusive_reason says why (below). It is never a pass.
skipped applies_when is false: the rule does not apply to this document.

inconclusive_reason: missing_field (the field is absent or has no value), type_error (the value cannot be converted, e.g. text where a number is needed), expression_error (a CEL runtime error or an internal error), model_unavailable (the AI judge could not answer), connector_unavailable, connector_timeout and connector_rate_limited (external validators) and no_docir_table (row_count against a DocIR table that does not exist). Any failure while evaluating a rule becomes an inconclusive verdict, never an exception.

Combinators use three-valued (Kleene) logic: all_of fails if any child fails, else it is inconclusive if any child is, else it passes; any_of passes if any child passes, else it is inconclusive if any child is, else it fails; not swaps pass and fail and keeps inconclusive.

A rule

{
  "id": "totals-add-up",
  "name": "Net plus tax equals the total",
  "kind": "deterministic",
  "severity": "error",
  "blocking": true,
  "applies_when": {"type": "comparison", "left": "currency", "op": "==", "right": {"literal": "EUR"}},
  "check": {"type": "arithmetic", "operands": ["net_total", "tax_total"], "operator": "sum", "equals": "gross_total"},
  "version": 1,
  "on_inconclusive": "review"
}
Field Notes
kind deterministic (uses check), ai (uses prompt and source_field_ids) or external (uses external).
severity error, warning or info; copied to the result.
blocking When true, a fail makes has_blocking_failure(results) true (review is mandatory before exporting).
on_inconclusive review (default) or ignore. A blocking rule that is inconclusive requires review (requires_review(results, rules)) unless it says ignore.
applies_when Any check. False → skipped; inconclusive → inconclusive; true → the rule is evaluated.
source_field_ids The fields a rule is about: their evidence is attached to the verdict (and, for AI rules, they are the only data the judge sees).

The JSON Schema of a rule is schemas/rules-1.0.json (checked by make contracts); see examples/rules/ for complete rule sets.

Checks

A field reference is a field_id, a path (supplier.vat_id), a row path (line_items[*].amount) or a field name. load_rules stores field_id (a concrete row such as line_items[2].amount stays a path). A [*] reference is evaluated per row: the check fails if any row fails and is inconclusive if some row is and none fails.

Type Parameters Passes when
required field the field is present, not not_found/abstained, not empty.
range field, min?, max? min <= value <= max (inclusive). Numbers may be tolerant strings (1.234,56, € 12,50, 12 %).
date field, earliest?, latest? the ISO date is inside the bounds; a bound is an ISO date or today (injectable clock).
arithmetic operands, operator (sum, subtract, product), equals, tolerance=0.01 the operation equals equals within an absolute tolerance. A [*] operand counts as the sum of its rows. Detail: 4176,93 + 417,69 = 4594,62 (tolerancia 0,01).
comparison left, op (==, !=, >, >=, <, <=), right ({"field": …} or {"literal": …}) the comparison holds. Numbers, ISO dates and text compare as such; booleans only with ==/!=.
one_of field, allowed, case_sensitive=true the value (every element of a multi-select) is in allowed.
regex field, pattern the RE2 pattern matches (partial match, like JSON Schema: anchor with ^…$). RE2 runs in linear time, so a pattern such as (a+)+$ cannot hang the engine; look-around and back-references are not supported.
checksum field, algorithm (iban, luhn, nif_es, vat_eu, isbn, lei) the check digit is valid. nif_es covers DNI, NIE and CIF; vat_eu is the EU VAT number with country prefix.
confidence field, op=">=", threshold (0–1) the field confidence satisfies the comparison.
evidence field, at_least (verified or partial) the best non-contradicting evidence is at least that verified.
row_count field, against (declared_field, docir_table or table_stats), count_field?, logical_table_id?, table? the extracted rows equal the count printed in the document (count_field), or the rows of the DocIR logical table (without that table: inconclusive / no_docir_table), or — table_stats, declared implicitly for x-dx.table.completeness = all_rows (T22) — the rows the table extraction accounts for (rows_physical - rows_excluded - rows_merged of table_stats[table]): fail with faltan N filas when some are missing.
expression cel the CEL expression is true (see below).
all_of, any_of, not checks / check Kleene combination of the children.

required is the only check that fails on a missing field; every other check is inconclusive / missing_field when the field is absent, not_found or abstained, and inconclusive / type_error when the value cannot be converted.

CEL expressions

expression checks use CEL (cel-python) over a variable data keyed by field_id. Tables are lists of objects whose keys are the field_id of their columns; the macros map, filter, exists and all come with the interpreter. The engine adds three functions:

  • num(x) converts numbers and strings (1.234,56, 1,234.56, € 12,50, -12.5, 12 %) to a double and raises a typed error for null, booleans and non-numeric text.
  • sum(list) adds with num (an empty list is 0).
  • abs(x) applies num and takes the absolute value.
abs(num(data.opening_balance)
    + sum(data.movements.map(m, num(m.credit)))
    - sum(data.movements.map(m, num(m.debit)))
    - num(data.closing_balance)) <= 0.01

Write expressions with field names: load_rules rewrites them to field_id (string literals are never touched), and canonicalize_expression(cel, schema, to="names") turns a stored expression back into names for an editor. An unknown field is a load error with a suggestion (unknown_field_ref).

Runtime errors: a null reaching num is missing_field; a missing key, a division by zero (also for doubles) or any other error is expression_error; a result that is not a boolean is type_error. Limits: 4 KiB of text, 32 levels of nesting and 10 000 rows per table. Compilation is cached by canonical text. A simple precompiled expression runs at several thousand evaluations per second per core (pytest -k bench_cel).

AI rules

{
  "id": "supplier-is-a-company",
  "kind": "ai",
  "prompt": "The supplier must be a legal entity (a company), not an individual person.",
  "source_field_ids": ["supplier_name"]
}

The judge never sees the document: only the values and short citations of source_field_ids, inside a block delimited by a token derived from the content (a value cannot close the block) with a system prompt that tells the model to treat it as data. The answer is {status, detail, cited_fields} with purpose="ai_rule" and temperature 0. cited_fields is filtered to the fields that were provided (the rest are dropped and logged), the call id goes to model_call_id, and answers are cached by hash of (rule text, values) for the lifetime of the gateway. If the model is missing or fails the verdict is inconclusive / model_unavailable. estimate_ai_rule_cost(rules, n_docs) estimates calls and tokens for an editor.

from extractit.llm.registry import resolve_gateway

results = evaluate(rules, view, gateway=resolve_gateway())  # DX_MODEL=codex, gemini, fake…

External validators

A rule with kind = "external" calls a validator registered by name (VIES, GLEIF, a bank registry…). The engine substitutes ${field_id} in params with the normalized value of the field (a missing field is inconclusive / missing_field) and translates the answer:

ExternalResult.status Verdict
ok pass
invalid, not_found fail
unavailable, timeout, rate_limited inconclusive with connector_unavailable, connector_timeout, connector_rate_limited
validator not registered, or it raised inconclusive / connector_unavailable

The network call, caching, rate limits and SSRF protection belong to each validator. A minimal one:

# doc-test
import asyncio

from extractit.core.ports import ExtractionView
from extractit.core.results import ExtractedField
from extractit.rules import (
    ExternalResult,
    ExternalValidatorRegistry,
    ValidationRule,
    evaluate,
)


class AlwaysActive:
    name = "demo"

    async def call(self, operation, params, ctx):
        await asyncio.sleep(0)  # a real validator awaits its HTTP client here
        return ExternalResult(status="ok", source_ref="demo:" + params["vat_number"])


registry = ExternalValidatorRegistry()
registry.register(AlwaysActive())
rule = ValidationRule.model_validate(
    {
        "id": "vat-active",
        "kind": "external",
        "external": {"validator": "demo", "operation": "check", "params": {"vat_number": "${vat}"}},
    }
)
view = ExtractionView(
    [ExtractedField(path="vat", field_id="vat", value="ESA58818501", status="extracted")]
)
(result,) = evaluate([rule], view, registry=registry)
assert result.status == "pass"

Packages register validators under the entry point group extractit.external_validators (a class with a no-argument constructor, or an instance); ExternalValidatorRegistry(entry_points=True) loads them.

Evidence

Every result has compared_values (what was compared) and evidence_ids: the sorted, deduplicated union of the evidence of the rule's source_field_ids, the fields its check (or applies_when) reads — for CEL, every field the expression names — and the fields an AI judge cited. At most 200 ids are kept; when some are dropped detail ends with (+N evidencias).

API

  • evaluate(rules, view, *, gateway=None, registry=None, clock=None, docir=None, cache=None) and evaluate_async(...): one ValidationResult per rule, in order. Deterministic rules never start an event loop; evaluate also works inside a running loop. docir (a ParsedDocument) feeds row_count against DocIR tables.
  • RulesValidator(gateway=…, registry=…, clock=…, docir=…) implements ValidatorPort (evaluate(rules, view)); it is registered as the entry point rules of the group extractit.validators.
  • load_rules(path_or_list, schema) resolves references and compiles CEL; it raises RuleLoadError with every issue (unknown_field_ref, ambiguous_field_ref, field_ref_not_leaf, syntax_error…) and a suggestion when there is one.
  • has_blocking_failure(results) and requires_review(results, rules).