ZoidLab DataForge — Foundry Package 15

Synthetic Data Studio. The Foundry app that feeds the other Foundry apps — it answers "where do I get labeled data to benchmark, evaluate, and ground with?" by generating real synthetic datasets through the live Nyquest relay, then validating every record with deterministic code — the model generates, the code checks.

Live at dataforge.zoidlab.ai. Part of the ZoidLab Foundry. Requires Nyquest Pro (enforced on both the frontend gate and every backend data endpoint, fail-closed).

What it does

• Generators — a spec for the data you want: a kind, a schema, a style, a domain, and a default count. Three kinds:
  • classification — {text, label} where label is constrained to a label set.
  • qa — {question, answer} reference pairs.
  • structured — rows matching a column schema (string|number|integer|boolean|date).
• Real generation — one synchronous relay call producing a JSON array of records (temperature 0.4, count capped at 100). Latency is wall-clock measured; tokens come from the relay's usage.
• Deterministic validation — labels must be in the set, required columns present, types checked in plain Python. Each record carries a valid flag and per-record errors — never a model grading its own output.
• Export where it's needed — datasets export as a signed Foundry envelope, plus convenience shapes that drop straight into:
  • ModelBench datasets (prompt + expected) — classification & QA,
  • Eval test sets (input + expected/reference) — all kinds,
  • RAG QA sets (question/answer pairs) — QA. Only records that passed validation are placed in the downstream shapes.

Honesty

• Every record comes from a real relay call — nothing is templated or simulated. Seeds contain generator definitions only, never generated records (no dataset is seeded).
• Validation is code, not vibes: a record is valid only if it matches the requested schema/labels.
• cost_usd is stored only when the relay reports it in usage; it is never invented from a local price table (it shows — otherwise).
• If no relay key is configured, /api/generate returns 503 relay_unavailable — it never fabricates output.

Stack

• Backend: FastAPI on the shared foundry-common platform layer — auth.py, entitlements.py, envelope.py, llm.py are thin shims; db_pg.py uses foundry_common.db (Postgres pool + per-tenant Row-Level Security on generators / datasets).
• Frontend: Next.js 15 + React 19 + Tailwind. Shared zb_session SSO + reusable Pro gate (Next middleware, <FoundryAccessGate>, and backend require_pro — three layers, fail-closed).
• Deploy (zoidberg): dataforge-api (:8709) + dataforge-web (:3709) behind the Cloudflare tunnel at dataforge.zoidlab.ai.

Dev

cd backend && python -m venv .venv && .venv/bin/pip install -r requirements.txt
NYQUEST_API_KEY=... .venv/bin/uvicorn main:app --port 8709
cd ../frontend && npm install && npm run dev   # proxies /api → 127.0.0.1:8709

Requires the shared Foundry Postgres (RLS roles foundry / app_rls) — point DATABASE_URL and DATABASE_URL_ADMIN at a dataforge database. Set MOCK_PRO_USER=true for local dev without a Nyquest session.