✦ Reproduce it

Run ScaffBench yourself

The harness is open source. Clone it, point it at any agent — Claude Code, Codex, opencode, Kilo, or Antigravity for Gemini — and it scaffolds each spec, then scores whether the generated project actually installs and builds. Runs work with a logged-in CLI or a plain API key.

QuickstartBrowse the reports

✦ Quickstart

Three steps

  1. 1Clone & install
    clone the harness
    $ git clone https://github.com/Marve10s/Better-Fullstack.git$ cd Better-Fullstack$ bun install
  2. 2Authenticate your agent

    Use an agent CLI you're already signed into (subscription / OAuth). Log in once, then the harness drives it — no keys in your environment.

    sign in to your agent
    # Claude Code (Anthropic) — sign in via the app, or:$ claude /login # Codex (OpenAI)$ codex login # Antigravity (Gemini)$ agy   # sign in on first launch
  3. 3Run the benchmark
    run all 13 specs, prompt path
    $ bun run scaffbench:2 --model claude-opus-4-8 --efforts max --paths prompt

    Prefer to keep validation clean? Split it into two phases — generate everything first, then validate on its own:

    two-phase
    $ bun run scaffbench:2:generate --model gpt-5.5 --paths prompt --out-dir runs/gpt55$ bun run scaffbench:2:validate --out-dir runs/gpt55

    Results — a leaderboard, per-spec pass, wired-libraries, and cost — land in the output directory, in the same shape as the published reports.

✦ Agents & models

Bring any agent

The provider is inferred from the model id, so one flag picks both the model and the CLI that drives it.

AgentExample modelsAuth
Claude Codeclaude-opus-4-8, claude-sonnet-5, claude-sonnet-4-6subscription · ANTHROPIC_API_KEY
Codexgpt-5.5, gpt-5.3-codex-sparkOPENAI_API_KEY
Antigravity (agy)gemini-3.5-flash, gemini-3.1-proGoogle sign-in
opencodeopencode/<model> (incl. free tier)opencode login
Kilo Codekilo/<provider>/<model> (incl. free tier)kilo login

✦ Flags

Tune the run

  • --model <id>the model to run (see the table above); the provider is inferred from the id
  • --efforts <tier>reasoning effort, where the model supports it
  • --paths prompt|mcp|cliprompt hand-writes everything; mcp goes through the MCP tools; cli composes the CLI command
  • --specs corethe full 13-spec suite by default, or a comma-separated subset of spec ids
  • --generate-only / --validate-existingsplit the run into a generate phase and a validate phase, validated on its own
  • --out-dir <path>where results land; re-use the same directory to resume or validate

✦ Compare

See how your run stacks up

Your numbers land in the same format as the leaderboard. Ran something interesting? Open a pull request with your report.

View the leaderboardRead the writeup
Patreon