Skip to content

Code node (transform)

The transform node (Code in the Studio) runs a short Python function you write over what reaches it in the workflow and stores the result as a downloadable JSON artifact.

def transform(input_data: Input) -> Output:
    lines = input_data.line_items or []
    return Output(lines_sum=round(sum(l.amount or 0 for l in lines), 2), lines=len(lines))

Input and Output are Pydantic models built for you:

After Input
extract the extraction's data: field names in snake_case, every value may be None, no confidence or evidence
parse pages: [{number, text, blocks: [{id, type, text}]}]

Output comes from the node's output_schema (the extraction-schema dialect of T02: string, number, integer, boolean, enum, array, object, nullable types). The result is validated twice — by Pydantic inside the sandbox and by jsonschema outside it — and a violation fails the node with SANDBOX_OUTPUT_INVALID and the JSON pointers of the offending values. transform may be async.

What the code can use

A closed list of modules (ADR-015): the pure standard library (math, decimal, datetime, json, re, collections, itertools, statistics, random, hashlib, …), pydantic, python-dateutil and regex. Anything else — requests, os, subprocess, socket, importlib, js — raises ModuleNotFoundError. python-stdnum is not on the list (LGPL-2.1).

Limits

Limit Default Error
wall time and CPU time 60 s (limits.timeout_seconds) SANDBOX_TIMEOUT
memory (resident) 1 024 MiB (limits.memory_mb) SANDBOX_MEMORY
result size 1 MiB SANDBOX_OUTPUT_TOO_LARGE
source code 64 KiB rejected when the workflow is validated

A workflow is also rejected at validation when the code does not parse (ast.parse), does not define transform, or when output_schema is not a valid extraction schema (transform_code_too_large, transform_syntax_error, transform_missing_function, transform_output_schema_invalid).

Failures

A failed transform ends the node failed without retry (one attempt). The public error has only a stable code and the exception_type (ValueError…); the traceback and whatever the code printed stay in the internal column node_runs.error_detail, never in the API.

Code When
SANDBOX_USER_ERROR the code raised
SANDBOX_TIMEOUT, SANDBOX_MEMORY, SANDBOX_OUTPUT_TOO_LARGE, SANDBOX_OUTPUT_INVALID limits and output contract
SANDBOX_UNAVAILABLE deno or the Pyodide bundle is missing, or the container runtime is
SANDBOX_PROFILE_INSUFFICIENT DX_DEPLOYMENT=saas with DX_SANDBOX=deno
CODE_NODE_DISABLED DX_CODE_NODE_ENABLED=0

How it is isolated

Each execution starts a fresh Deno process that loads Pyodide (CPython compiled to WebAssembly) from a local bundle:

  • Deno runs with --deny-net --deny-run --deny-ffi --deny-env --deny-sys --deny-write, read access only to the bundle and the runner, and read access revoked before your code runs. Python in Pyodide can reach JavaScript (import js), so the boundary is Deno's permissions, not Python: fetch, Deno.readTextFileSync, Deno.env, Deno.Command all fail with NotCapable, and sockets fail with OSError.
  • The process gets an empty environment (only PATH and an ephemeral DENO_DIR), an ephemeral working directory and its own process group. A guardian samples its memory (physical footprint on macOS, RSS + swap on Linux) and CPU every 50 ms and kills the whole group on a breach.
  • DX_SANDBOX=deno+gvisor (required in multi-tenant SaaS) runs the same command inside an ephemeral container per execution with gVisor (runsc), --network none, a read-only root, --cap-drop ALL, no-new-privileges, 64 processes, 1 GiB and 1 CPU, as user 65534; the image (sandbox-runner, built from src/extractit/nodes/code/sandbox/Dockerfile) is pinned by digest in DX_SANDBOX_IMAGE. The Kubernetes launcher produces the equivalent Job (runtimeClassName: gvisor, deny-all NetworkPolicy, activeDeadlineSeconds); it is validated on a cluster in T53.

Installing the sandbox

$ python -m extractit.nodes.code.install            # Pyodide bundle into ~/.extractit/sandbox
$ python -m extractit.nodes.code.install --with-deno # also the pinned Deno binary
$ python -m extractit.nodes.code.install --check

Versions and sha256 of every file are pinned in src/extractit/nodes/code/sandbox/pins.toml (Pyodide 314.0.7, Deno 2.9.8). Nothing is downloaded at run time; an air-gapped install copies the directory (DX_SANDBOX_BUNDLE, DX_DENO_PATH).

A cold start (Deno + Pyodide + pydantic) takes about 1.5 s on a laptop (p95 of 20 runs); the CI budget is 6 s.