Skip to content

Extending the pipeline

Two registries let later tickets and third-party packages add capabilities without touching the core (extractit.extract.extension). Results of both go through the same assembly rules as the harness: field states, heuristic confidence, review_state and requires_review.

Field handlers: register_field_handler

A FieldHandler replaces the extraction harness for the schema fields it applies to — tables (T22), long documents (T23), visual elements (T56).

from extractit.core.results import FieldCandidate
from extractit.extract.extension import HandlerResult, register_field_handler


class TotalsBox:
    kind = "visual"

    def applies(self, field, ctx):  # FieldDef, PipelineContext
        return field.name == "total"

    def extract(self, fields, ctx):
        (total,) = fields
        line = next(ln for ln in ctx.doc.lines if "Total:" in ln.text)
        return [
            HandlerResult(
                candidate=FieldCandidate(
                    path="total",
                    field_id=total.field_id,
                    status_hint="extracted",
                    value=372.0,
                    raw_text="372.00",
                    anchors=(line.anchor,),
                )
            )
        ]


register_field_handler(TotalsBox(), priority=50)
  • Handlers are tried in ascending priority (default 100), then registration order, on every schema field, containers before their children; a claimed container (e.g. line_items) takes all its descendants. The harness is asked only for the remaining fields (only_field_ids).
  • HandlerResult(candidate, evidence=None, status=None, signals=None): without evidence the candidate is normalized and verified by T07 like any harness answer; with evidence (for instance table cells) the generic verification is skipped and the handler's evidence is used. status forces the field state; signals seeds the confidence signals.
  • When the harness raises DocumentTooLongError, the pipeline hands the fields to the handler of kind = 'long_document' (it need not apply to anything); without one the error is raised as PipelineError(step='extract', kind='input').

Source kinds: register_source_kind

A SourceKind resolves the fields whose x-dx.source is not document — lookup, derived, metadata (T24) or visual/external (T56).

from extractit.extract.extension import ResolvedField, register_source_kind


class Doubled:
    kind = "derived"
    phase = "post_verify"  # sees ctx.view with the verified fields

    def resolve(self, fields, ctx):
        total = ctx.view.value("total")
        return [
            ResolvedField(field_id=f.field_id, value=total * 2, status="derived") for f in fields
        ]


register_source_kind(Doubled())
  • phase = 'pre_extract' runs right after parsing; 'post_verify' runs after verification with ctx.view (an ExtractionView of the other fields).
  • ResolvedField(field_id, value, raw_text, status, evidence, signals, path=None); path defaults to the field's own path.
  • A field whose source has no registered resolver is abstained with the warning source_kind_unregistered.

PipelineContext

doc (DocIR), am (AnchoredMarkdown, None for a document without text), schema, gateway, ledger (the gateway's call ledger), options (PipelineOptions), tier, view (post-verify resolvers) and warn(code, message, path=None) to add an output warning (never put document text in it). If a ticket needs more, the change is additive.

Entry points

Installed packages register automatically through the entry point groups extractit.field_handlers and extractit.source_kinds. The object may be the handler itself or a zero-argument factory such as the class; a handler's priority attribute is honoured.

[project.entry-points."extractit.field_handlers"]
tables = "dx_tables:TableHandler"

[project.entry-points."extractit.source_kinds"]
lookup = "dx_lookup:LookupSource"

Entry points load once per process on first use; a plugin that fails to load is logged and skipped. Registries are global to the process: tests call register_* / unregister_* or reset_registry().