Skip to content

Security gates

ExtractIt has no remote CI and no required checks on main (decision of the maintainer, 2026-11-12). The gates below are local make targets and pytest markers; nothing here is enforced by a server. A release is only as safe as the last time somebody ran them.

Gate Command What it proves Discovers by Needs
ASVS list make security-asvs security/asvs.yaml is one-to-one with the official ASVS 5.0.0 JSON (pinned by sha256), every verified entry names evidence and tests that exist, no L1 requirement of an assessed chapter is planned, no high/critical finding is open the official JSON —
Injection make security-injection Obedient fake model over 66 adversarial documents (6 classes): 0 tool calls, 0 connections; tool-hijack against every discovered registry (110 attempts); a purpose of a model call without a corpus case fails; CSV/XLSX writers quote-prefix formulas; the viewer renders hostile values as text; SSRF matrix of 103 destinations AST of src/ (purposes, registries, outbound calls), writers and the OpenAPI viewer e2e needs pnpm install + Playwright
RLS make security-rls Every table with tenant_id has RLS enabled and forced, a policy with USING and WITH CHECK, and a tenant cannot read or write another tenant's rows (also with an empty context); the app role is not superuser nor BYPASSRLS pg_catalog Postgres
API leak make security-api-leak Every operation of openapi/openapi.json is called as tenant B with tenant A's ids: 403/404, never 2xx or 5xx, no identifier of A in an answer, A's rows unchanged the OpenAPI snapshot Postgres
Sandbox make security-sandbox 42 escape attempts against the Code node sandbox (Deno + Pyodide) are refused with the expected error; no Deno process is left behind a fixed list Deno + the Pyodide bundle
make security all of the above the union

Docker-based and on-demand gates (not in make security; they need Docker and several minutes):

Gate Command What it proves Needs
ZAP make security-zap zap-api-scan.py (official zaproxy/zap-stable image, Apache-2.0) over every operation of openapi/openapi.json (sliced by tools/zap_spec.py, 25 paths per scan), authenticated with the API key of the local stack, plus zap-baseline.py; tools/zap_verdict.py fails on any High or Critical alert; accepted Medium alerts go in security/zap/rules.tsv with a reason Docker, DX_STACK_INFO of tests/load/stack.py running
Load make load Locust (MIT, run through uvx) at 100 docs/min for 5 min with a fixed-latency model: error rate < 0.1 %, 0 lost runs; then the 1 000 burst (202, all terminal < 15 min) and 50 submissions with one Idempotency-Key (1 run, 49 replays) Postgres
Chaos make chaos 30 s of latency, then 60 s without Postgres (Toxiproxy) while 30 runs are submitted: all succeed, none lost, no step with two successes Docker, Postgres

Also run by hand: bandit -r src ee/src -lll, pip-audit (make audit), gitleaks detect (Docker image zricethezav/gitleaks; the per-file allowlist is in .gitleaks.toml), pnpm audit --audit-level high.

How a gate fails on something new

  • A table with tenant_id and no row factory (tests/tenancy/factories.py or tests/security/tenant_leak/fixtures_factory.py) fails security-rls.
  • An operation whose path parameter cannot be harvested as tenant A, with no @resource and no line in EXEMPT_OPERATIONS, fails security-api-leak (statically, with no database).
  • A component that sends a model request under a new purpose fails security-injection until tests/redteam/purpose_coverage.py has an exemption with its reason or the corpus reaches it.
  • A new module-level tool registry (ALLOWED_TOOLS, *_REGISTRY, TOOLS) fails until it is in KNOWN_REGISTRIES of tests/redteam/tool_registries.py.
  • A new outbound call site fails tests/security/ssrf until it is registered with its guard.
  • A new export format (found through the writers and the OpenAPI) is run through the 100 payloads of the formula gate.

NFR-08 (corrupted values stopped)

The strict NFR-08 gate (95 % of the corrupted values end unverified, contradicted or fail a rule) is green and blocking since FIX-T07: 100 % (45/45). T55 measured 82.2 % (37/45); the eight misses were findings SEC-005, SEC-007 and SEC-008, fixed in the T07 verifier. The list of misses is pinned (empty), so a new miss fails the gate. A second gate is green and blocking: no corrupted value ends in the high confidence band, with no exception.