Skip to content

Hardening report (T55)

State: PARTIAL. Verified on a cloud VM (4 vCPU, 15 GB, Docker): the leak gates, the injection suite, SSRF, formula injection, the viewer XSS run in Chromium, ZAP (API scan and baseline), Locust (5 min), the burst and idempotency scenarios, a local chaos run (database latency and outage behind Toxiproxy) and the supply-chain scanners. Not run: the sandbox battery (Pyodide bundle is unreachable from the VM), the Compose and kind chaos, the 30-minute staging load, gVisor, the ASVS chapters other than V8. The NFR-08 criterion "95 % of corrupted values stopped" was not met by T55 (82.2 %) and is met since FIX-T07 (100 %, 45/45). The full table criterion -> test -> result of the cloud run is in DELIVERY-T55.md.

All gates are local (make security*, see Security gates): there is no remote CI.

Criteria

Criterion State Evidence
RLS on every tenant table, discovered from pg_catalog met make security-rls: 12 tests, including negatives (no RLS, no FORCE, no WITH CHECK, no factory, permissive policy, BYPASSRLS, empty context)
Every API operation as tenant B leaks nothing met make security-api-leak: 175 operations of openapi/openapi.json walked with tenant A's ids; 403/404, no identifier of A, A's rows unchanged; one exemption (segment PDF, reason in EXEMPT_OPERATIONS)
Injection suite: 0 tool calls, 0 connections met tests/redteam: 66 documents in 6 classes; request has no tool channel; socket guard saw 0 attempts
≥ 95 % of corrupted values unverified/contradicted/rule-failed met: 100 % (45/45) since FIX-T07 (T55 measured 82.2 %, 37/45) test_at_least_95_percent_of_corrupted_values_are_unverified_contradicted_or_fail_a_rule (no xfail); test_the_misses_are_exactly_the_documented_findings pins an empty list
No corrupted value in the high confidence band met, no exception test_no_corrupted_value_reaches_the_high_confidence_band
A purpose of a model call without a corpus case fails met tests/redteam/purpose_coverage.py (AST); 9 exemptions with reasons
Tool hijack against discovered registries met tool_registries.py: 110 attempts, none resolves to a tool; every DELETE and publish/alias operation is x-dx-destructive
CSV/XLSX formula injection met 0 of 100 payloads survive; XLSX needed a fix (quote prefix)
SSRF on webhook and URL destinations met 103-destination matrix, outbound-surface registry; SEC-004 fixed
Viewer renders hostile values as text met packages/viewer/e2e/xss.spec.ts: 62 payloads (event handlers on 8 tags x 6 sinks, javascript: links, markdown, <base>, exfiltration images), Chromium: no script, handler, dialog or outside request
Sandbox escape battery ≥ 30 met 42 attempts + 3 side-channel checks, deno profile; container profile not run
H8 confirmed confirmed for the deno profile ADR-015
ASVS 5.0.0 list checked on every run partial security/asvs.yaml: 345 requirements, V8 assessed (5 verified, 2 planned, L3 planned), the other chapters planned under SEC-ASVS-PENDING
ZAP API scan + baseline met (limits below) make security-zap: 175 operations in 6 scans, 0 High/Critical; baseline found X-Content-Type-Options missing on API answers (SEC-012, fixed)
Load: 5 min at 100 docs/min met 500 runs, 499 succeeded and 1 request connection error at the end of the window (0.027 % < 0.1 %), 0 lost, end-to-end p95 6.7 s
50 submissions, one Idempotency-Key met 1 run, 49 Idempotent-Replayed: true
Burst of 1 000 partial 1 000 x 202 and all terminal in 449 s (< 15 min); submit p95 4.4 s with 40 concurrent clients (criterion 2 s) and 1.4 s with 10 (200 documents); single-process stack on 4 vCPU (SEC-013)
Chaos: database 30 s latency + 60 s outage met locally make chaos: 30/30 runs succeed, 0 lost, 0 steps with two successes (after SEC-017: 4 of 4 runs; before it 2 of 5 had one failed run). Raw reports: docs/security/reports/
Chaos on Compose (worker kill, provider cut), kind not run script written (tests/chaos/compose_chaos.sh); the image cannot be built here
500-page document, 25 MB file, 30 min staging load not run parse fake knows one document; no staging
bandit, pip-audit, gitleaks, pnpm audit met 0 high: see findings SEC-014..016
External pentest pending action Pentest checklist

Findings

Id Severity Finding State Owner
SEC-004 medium SSRF in the per-organization SSO issuer discovery (no address guard) fixed (ee/tests/test_ee_sso_ssrf_unit.py) T49
SEC-005 medium An injection that quotes the literal value is verified against its own instruction line fixed (FIX-T07: lines of an instruction addressed to the model are never evidence, ground/instructions.py; tests/ground/test_verifier_injection.py) T07
SEC-006 low http://localhost accepted as SSO issuer without a flag (development exemption) open T49
SEC-007 medium An enum value the verifier cannot read (currency=USD) ends block_only (confidence 0.60), not unverified: 4 of the 8 misses fixed (FIX-T07: block_only only when the normalizer reads the option in a phrase of the cited text; tests/ground/test_verifier_injection.py) T07
SEC-008 medium Fuzzy partial grounding accepts a short value that shares a token with the cited line (CALLED read_file): 3 of the 8 misses fixed (FIX-T07: a fuzzy match must read ≥ 80 % of the value's characters as whole words, ground/matching.py::word_coverage; tests/ground/test_verifier_injection.py) T07
SEC-009 low GET /v1/workflows/{id}/documents/columns answered 200 (empty) for a workflow of another tenant; PUT already answered 404 fixed, found by the API-leak walk T55
SEC-010 info Deno.cwd() answers inside the sandbox (names only the throw-away directory) accepted T30
SEC-011 info os.system crashes the sandbox runner and a Worker hangs to the wall limit (both fail closed) open T30
SEC-012 low API answers had no X-Content-Type-Options: nosniff (ZAP baseline) fixed (api/middleware.py, tests/api/test_api_conventions.py) T55
SEC-013 low Submit p95 4.4 s at 40 concurrent uploads on one process (4 vCPU, worker in process); PG pool 32 with no overflow timed out at 100 concurrent open (sizing; staging run needed) T53
SEC-014 high (dev tool) js-yaml < 4.3.2 under @hey-api/openapi-ts (3 high advisories, build-time only) fixed (pnpm override js-yaml >=4.3.2) T55
SEC-015 low gitleaks: 5 fake sentinels in test fixtures from develop fixed (per-file allowlist in .gitleaks.toml, 0 leaks) T55
SEC-016 info bandit B701 (Jinja autoescape off) in two Markdown renderers accepted (# nosec, not HTML) T55
SEC-017 medium 60 s without Postgres exhausted the 3 attempts of the persist step: the run ended failed (found by the chaos run, 2 of 5 runs of 30) fixed (runtime/steps.py: database-unreachable failures retry up to 9 attempts, ~2 min; 4 of 4 chaos runs green after) T55
XLSX medium Cells starting with =, +, -, @, tab or CR were written as formulas fixed (quotePrefix; tests/security/formula_injection) T55
x-dx-destructive low The mark did not exist: every DELETE and publish_workflow_version/set_workflow_alias now carry it fixed T55

The eight corrupted values the strict criterion missed at T55: visible-12 (SEC-005); image_ocr-04, invisible-03, invisible-11, visible-06 (SEC-007); tool_hijack-06, -09, -11 (SEC-008). FIX-T07 fixed the three findings in the T07 verifier and the list is now empty: a test pins it, so any new miss fails it. No legitimate value of the repository's fixtures moved from verified to a worse state (verdict-by-verdict comparison of tests/ground, tests/eval, tests/golden, tests/redteam, tests/extract, tests/pipeline, tests/review, tests/nodes, tests/knowledge, tests/agent_golden and tests/rules against develop c47ec6ca). Limit: the instruction markers are a lexicon; an instruction phrased without any of them that also quotes the value would be grounded (see Verification).

Not done (declared, not run)

  • Sandbox battery (make security-sandbox): skipped on the cloud VM (46 skipped: the Pyodide bundle is on GitHub/jsDelivr, blocked by the VM proxy). It passed on the maintainer's laptop (46). Container profile (gVisor): never run.
  • Compose and kind chaos (kill the worker, cut the provider): tests/chaos/compose_chaos.sh is written and untested; the image needs the PyTorch CPU wheels (unreachable here).
  • Staging: the 30-minute load, the 500-page document and the 25 MB file.
  • ZAP depth: ZAP substitutes placeholder ids, so most requests answer 400 (the 604 client errors are all 400, none 401: the key works). Authenticated deep paths are covered by the leak gates, not by ZAP. The Studio (SPA) is not served by the local stack: CSP and frame-ancestors come from web_static.py and viewer e2e (CSP without violations), not from the baseline.
  • ASVS: chapters other than V8 are planned (SEC-ASVS-PENDING, owner: T55 follow-up).
  • Locust is not a project dependency (uv lock cannot reach the PyTorch index from the VM): make load runs it with uvx. Add it with uv add --dev locust where the index is reachable.
  • Python license check of the runtime group cannot run here (needs the torch wheels); the web check passes.
  • Videos and screenshots of the DoD: not recorded.