A reference implementation showing how to use LangGraph to build a multi-agent code generation pipeline with locally-hosted LLMs.
The pipeline generates complete Next.js App Router projects from a natural-language spec. It demonstrates several patterns that are useful when building LLM-powered code generation systems:
- Multi-model orchestration — a "foreman" model (gpt-oss 20B) handles planning and review, while a "coder" model (qwen3-coder) handles generation and fixes
- Decomposed generation — instead of asking one model to produce an entire project in a single prompt, the pipeline breaks the task into discrete nodes: architecture policy, file manifest, manifest review, code generation
- Automated build-fix loops — when
pnpm buildfails, a reviewer diagnoses the error and a coder applies targeted fixes, with spin detection to avoid infinite loops - Mechanical error recovery — common build failures (missing packages, unused imports, config file issues) are fixed deterministically without burning LLM calls
- Failure-informed regeneration — when fixes aren't enough, the pipeline regenerates with error context from the failed attempt
Everything runs locally on Docker Model Runner — no API keys, no cloud dependencies, no per-token costs. It's a step towards bringing the build-test-fix discipline of CI/CD to LLM code generation.
policy (gpt-oss) → scaffold (create-next-app) → manifest (qwen3-coder) → review (gpt-oss)
→ generate (qwen3-coder) → write → install → build
↓
review (gpt-oss)
↓ ↓
fix (qwen3) → regenerate
Each box is a LangGraph node. The pipeline is defined in factory.py using StateGraph — edges between nodes are explicit, and conditional routing handles the build-fix-regenerate loop. State flows through the graph as a typed dict.
| Node | What it does |
|---|---|
| policy | Foreman produces an architecture contract: layout, entities, routes, library usage notes |
| scaffold | Runs npx create-next-app@14 to create a working skeleton with correct config |
| manifest | Coder plans which app-specific files to generate (pages, API routes, lib) |
| review_manifest | Foreman reviews the plan, trims unnecessary files |
| generate | Coder produces all app files in one call using a fence format, merged on top of scaffold |
| write / install / build | Deterministic steps: write to disk, pnpm install, pnpm build |
| review | Foreman evaluates build failures — decides: fix, regenerate, or give up |
| fix | Coder applies targeted patches guided by the reviewer's diagnosis |
The pipeline has several layers that handle failures, applied in order:
- Post-generate sanitizer — deterministic fixes before the first build (next.config.ts rename, missing root layout, missing
"use client"directives, import reconciliation) - Mechanical fixes — pattern-matched build errors fixed without LLM calls (missing npm packages, unused imports/variables, incomplete prop interfaces, missing
"use client"directives) - Reviewer-guided LLM fixes — foreman diagnoses the error, coder applies the fix
- Spin detection — if the same file is patched 2+ times, forces regeneration
- Failure-informed regeneration — error patterns from failed attempts are fed into the next generation prompt
From examples/blog_markdown.py — blog platform with markdown rendering, admin CRUD, and comments:
Total pipeline time: 266.3s
Generate attempts: 1 | Fix attempts: 1 | Build attempts: 2
Result: BUILD OK
Step Time Tokens tok/s Model Notes
--------------------------------------------------------------------------
policy 27.1s 361 13.3 gpt-oss
scaffold 3.1s 13 files
manifest 68.0s 640 9.4 qwen3-coder-next 14 planned
review_manifest 29.5s 86 2.9 gpt-oss
generate 95.5s 4010 42.0 qwen3-coder-next 24 files
install 1.6s OK
build 6.4s FAILED
review 3.6s 111 31.1 gpt-oss
fix 24.2s 716 29.5 qwen3-coder-next 2 files patched
build 7.3s OK
Build failure was 2 ESLint errors (let → const). Reviewer diagnosed in 3.6s, coder fixed in 24s.
Full sample output: examples/sample_output_blog.txt
- Docker Model Runner with GPU support (or any OpenAI-compatible endpoint at
localhost:12434) - Node.js 18+ (includes
npx, used by the scaffold step) and pnpm - Python 3.12+
Install Docker Model Runner with GPU support and pull the models:
# First-time setup (or reinstall to enable GPU)
docker model install-runner --gpu cuda
# If already installed without GPU, reinstall:
# docker model uninstall-runner && docker model install-runner --gpu cuda
docker model pull ai/gpt-oss:20B
docker model pull ai/qwen3-coder-next:latestThe --gpu cuda flag ensures models run on your NVIDIA GPU instead of CPU. Without it, inference will be orders of magnitude slower.
Verify your setup:
docker model list
pnpm --version
curl http://localhost:12434/engines/v1/modelsgit clone https://github.com/t-espy/langgraph-factory.git
cd langgraph-factory
python3 -m venv .venv
source .venv/bin/activate
pip install -e .from langgraph_factory import build_factory_graph
graph = build_factory_graph()
result = graph.invoke({"spec": "A CRUD app for managing products...", "project_dir": "runs/my_run"})See examples/ for complete specs. Run all of them:
./run_all_specs.shTo run the unit tests:
pip install -e ".[dev]"
pytestAll config is via environment variables — see config.py for defaults:
| Variable | Default | Notes |
|---|---|---|
DMR_BASE_URL |
http://localhost:12434/engines/v1 |
Docker Model Runner endpoint |
FOREMAN_MODEL |
docker.io/ai/gpt-oss:20B |
Planning + review model |
CODER_MODEL |
docker.io/ai/qwen3-coder-next:latest |
Code generation + fixes |
MAX_GENERATE_ATTEMPTS |
2 |
Full regeneration budget |
MAX_FIX_ATTEMPTS |
4 |
Fix cycles per generation attempt |
BUILD_TIMEOUT |
120 |
Seconds before pnpm build is killed |
langgraph_factory/
├── factory.py # The pipeline — all nodes, routing, and recovery logic
├── llm.py # LLM client (streaming, retry, token stats)
├── utils.py # Fence parser, JSON extraction, logging
├── config.py # Environment-based configuration
└── __init__.py
examples/
├── blog_markdown.py # Blog with markdown rendering spec
├── crud_products.py # Products CRUD spec
└── sample_output_blog.txt # What a successful run looks like
tests/
├── test_factory_functions.py # Unit tests for recovery/validation logic
├── test_generate_only.py # Manual integration test (requires running LLMs)
└── test_utils.py # Unit tests for parsing and extraction
run_all_specs.sh # Run all example specs and collect results
This is a reference implementation — a worked example of how to wire up LangGraph, local LLMs, and deterministic tooling into a pipeline that produces working code. It demonstrates the patterns; it's not a production platform.
Things worth studying:
- How
factory.pydecomposes a complex task into bounded LLM calls - How the build-fix loop uses a reviewer model to guide a coder model
- How mechanical fixes avoid wasting LLM calls on deterministic problems
- How the scaffold node anchors generation on a known-good starting point
- How failure context flows back into regeneration prompts
The pipeline just talks to an OpenAI-compatible HTTP endpoint, so you can swap in different models or backends (Ollama, LM Studio, etc.) by pointing DMR_BASE_URL at it in config.py.
Development hardware was an NVIDIA DGX Spark (GB10 GPU, 128GB unified RAM) running Docker Model Runner. Observed throughput:
| Model | Parameters | Size (Q4) | tok/s |
|---|---|---|---|
| qwen3-coder | 80B | 45 GB | 40-44 |
| gpt-oss | 20B | 11 GB | 9-35 |
A typical successful run produces ~6,000 tokens across 4-6 LLM calls and completes in 4-5 minutes. Runs that need a fix cycle add ~1,000-4,000 tokens and 30-120 seconds.
This pipeline is token-heavy — a single run can consume 6,000-40,000+ tokens across multiple LLM calls, including retries and regenerations. That changes the economics compared to one-shot prompting.
Pay-per-token (cloud API): Using a comparable commercial model at ~$3/M input + ~$15/M output tokens, a clean 6K-token run costs roughly $0.05-0.10. But runs that hit regeneration (30K+ tokens) can reach $0.50-1.00+, and you'll have many of those while iterating on prompts and recovery logic. During development of this pipeline, we ran hundreds of generations — at cloud rates that adds up fast.
Rent a GPU (vast.ai, RunPod, etc.): An 80GB A100 rents for roughly $1-2/hr. You get unlimited tokens at whatever speed the hardware delivers. A 5-minute run costs ~$0.10-0.15 in GPU time regardless of token count, and regenerations are free. For iterative development work — where you're running the pipeline repeatedly to tune prompts and fix recovery logic — this is substantially cheaper than per-token pricing.
Own the hardware: The DGX Spark used here lists at $4,699 direct from NVIDIA. At $1.50/hr cloud GPU rates, it pays for itself after ~3,100 hours of usage. If you're running local models regularly, the amortized cost approaches zero.
Data privacy: With local or rented GPU, your prompts and generated code never leave your infrastructure. Nothing is sent to a third-party API, nothing is logged by a provider, nothing can end up in someone else's training data. For proprietary codebases, regulated industries, or anything you wouldn't paste into a web form, this matters. Per-token cloud APIs have varying data retention and training policies — read the fine print.
The broader point: pipelines like this generate a lot of tokens that never reach a user — reviewer reasoning, failed generations, fix attempts, regeneration context. Per-token pricing charges you for all of that, and sends all of it through a third party's servers. Local or rented GPU gives you a flat rate to experiment freely, with full control over your data.
This is a focused demo, not a comprehensive framework. Plenty of things that would matter in a real system are left as exercises for the reader:
- Other frameworks and languages — the pipeline generates Next.js because that's what we tested. The patterns (decompose → generate → build → fix) apply to any framework with a CLI build step. Swap out the scaffold command, adjust the prompts, and the same architecture works for Django, Rails, Spring Boot, or anything else with a compiler or linter to close the feedback loop.
- Persistent storage — everything uses in-memory stores. A real app would need database setup, migrations, and schema generation as pipeline nodes.
- Authentication and authorization — not even attempted. Adding auth scaffolding (NextAuth, Clerk, etc.) would be another node in the graph.
- Testing — the pipeline checks "does it build?" but doesn't generate or run tests. Adding a test generation node and a test runner after the build step is an obvious next step.
- Deployment — the pipeline produces a buildable project and stops. CI/CD, containerization, and deployment are out of scope.
- Model selection — we used two specific models because they were available and worked. The quality of the output depends heavily on the models. Better models will produce fewer fix cycles; smaller models may need more recovery layers or tighter prompt constraints.
- MCP integration — the pipeline talks directly to an OpenAI-compatible endpoint, so tool definitions and model routing logic live in the pipeline code. Wrapping the model server in an MCP layer would make those definitions shareable across clients — Claude Code, LangGraph, and future tooling could all call the same tools without duplicating the integration logic. The foreman/coder split in particular maps cleanly onto separate MCP servers with distinct capabilities, which would also make swapping models behind each role cleaner than editing config variables. On pay-per-token APIs, MCP's protocol overhead meaningfully inflates costs on short high-frequency calls like the reviewer; the economics favor MCP when running on local or rented GPU where tokens are effectively free.
- Prompt engineering — the prompts in
factory.pyevolved through trial and error on two specs. They encode specific workarounds for specific model behaviors (qwen3-coder's shadcn habit, marked() type issues). Different models will have different failure modes and need different prompt tuning. - Parallel generation — files are generated in a single monolithic LLM call. Generating files in parallel (one call per file or per group) would be faster but adds coordination complexity around shared types and imports.
- Streaming output — the pipeline blocks until each node completes. A production system would want to stream progress to a UI.
- Cost tracking — token counts are logged but not aggregated into cost estimates. Easy to add if you know your per-token rate.
The goal was to demonstrate the multi-agent pattern clearly, not to handle every edge case. Fork it, break it, make it do something different.