Unreleased
Changes merged and not yet in a release.
Added
- Self-service sign-up (
DX_SIGNUP=open, defaultclosed): «Continue with Google», «Continue with Microsoft» and «Email me a sign-in link» at/loginand/signup. The first verified sign-in of a new person creates their organization (Owner,defaultproject, welcome credits throughidentity.owner_verified) unless an invitation to that email is pending, which wins. Google and Microsoft Entra areDX_OIDC_ISSUERSpresets ("type": "google" | "microsoft"); multi-tenant Microsoft tokens are checked against their own tenant issuer (tid) and their email counts as verified only withxms_edov. Sign-in links:POST /auth/email/start(always 202),GET /auth/email/link(never spends it),POST /auth/email/confirm(once, 15 minutes, hashed token, per-email/address/global limits). One person, one account: identities with the same verified email join (user_identities, migration0031_t66).GET/PATCH /v1/organizationrenames the organization. Terms acceptance (DX_TERMS_VERSION) and the audit eventsauth.signup,auth.email_link_sent,auth.identity_linked. Seedocs/ops/signup.md. - Welcome credits (commercial edition,
DX_BILLING_MODE=saas): each new organization gets 1 000 promotional credits when its first Owner has a verified email (once per organization and per verified email, never to disposable domains). They only pay Flash-tier pages, are spent before purchased credits, expire after 30 days and are not refundable; a run on a higher tier with only welcome credits left is refused with 402credits-tier-not-covered.GET /v1/billing/balancenow returnspaidandpromo; settings (S15) and the Studio show them. Configured inplans.yaml(welcome_credits). New core hook pointidentity.owner_verified.
Changed
extractit FILEextracts directly (noextractverb); new short aliasxt; thedxalias is removed.- Core relicensed from Apache-2.0 to FSL-1.1-ALv2 (source-available; each release converts to Apache-2.0 after two years). SDKs, schemas, examples and skills stay Apache-2.0.
- Canonical URLs moved to extractit.dev (docs.extractit.dev, schemas.extractit.dev, extractit.dev/problems, errors.extractit.dev; SaaS host app.extractit.dev).
- Renamed: DataExtractor → ExtractIt (package, import, CLI with
dxalias, npm scope, images, charts). Tagline: «Extract your data. Every value cited.»DX_*variables,DX-Version, database roles, id prefixes, error-type URIs and JSON Schema$ids are unchanged.
Fixed
- Fields of type
arraywith simple elements (array<string>,array<number>,array<date>, nestedrows[*].tags[*]) no longer breakassemble(path indexes a non-array value) on the workflow and agent-mode paths:finalizeno longer adds anot_foundplaceholder for the whole array next to itsskills[0]…elements, anddata_from_fieldslets the elements win over such a placeholder. Elements ofarray<number|integer|date|datetime|boolean>are now normalized and verified by their element type (they were read as text, sodata.scoresheld strings). An absent top-level array is[]indataon every path. CSV/XLSX exports (and the column list) treat an array of scalars as one list cell, not as a one-column table. Output schema and DocIR are unchanged.
Added
-
T41 measured optimizer:
description_rewrite(1–10 rounds on the train/validation splits, candidates as drafts, acceptance by a paired bootstrap, one report on the sealed test split, USD budget, cancellation) andone_shot(unmeasured),/v1/optimizations, promotion by a person only, the Optimizer tab of Health (S11). Both strategies areexperimentaluntil a nightly evaluation shows a gain on test. -
T24 field sources:
x-dx.sourceas one object (document,lookup,derived,metadata; legacy strings normalize to it), reference tables with immutable versions (/v1/lookup-tables,dx lookup), CEL derived fields, citable package metadata, multi-file packets as one document with global pages, and external validators (VIES, GLEIF, bank registry) as rules with cache, rate limit and an SSRF-safe client. Contracts move to dialect 1.2, output 1.2 and DocIR 1.2 (additive; the 1.0 and 1.1 snapshots stay). -
Monorepo skeleton: Python package
dataextractor(uv, hatchling + hatch-vcs),dx --version, web workspace (Vite, React 19, TypeScript, Biome, Vitest, Playwright). make checkquality gate, identical locally and in CI.- Supply-chain controls: license allow lists, SPDX headers, pinned GitHub Actions,
pip-audit, Grype, gitleaks, DCO check. - Release workflow (PyPI/TestPyPI trusted publishing, multi-arch GHCR image, CycloneDX SBOM, keyless cosign signature, provenance attestation).
- Development Docker Compose stack (PostgreSQL 16, Gotenberg, OpenBao).
- Governance files: LICENSE (Apache-2.0), NOTICE, CONTRIBUTING, CODE_OF_CONDUCT, SECURITY.