prague-bench

prague-bench

A single-prompt benchmark for AI coding agents. Every model gets the same spec: build a procedural Three.js drone flight over Prague Castle and Charles Bridge, render it headless and deliver a 58-second vertical video. No human help along the way. Here is what each run cost, how long it took, how fast the model wrote, and what a blind panel of three AI judges thought of the result.

finished runs with a valid video
13
spent on these runs (list prices + billed)
$122
total wall time of these runs
14 h 6 min
latest blind panel, 11 videos
14 Nov

Leaderboard

The blind panel of 14 Nov 2026 judged 11 videos. Three judges from three vendors scored 16 frames of each video without knowing which model made it. The score is the median of the three.

Latest blind panel: judge scores, median and change since earlier panels
#ModelHarnessAnthropicOpenAIGoogleMedianSpreadvs 13 Novvs 11 NovH1–H6
1Claude Fable 5.2Anthropic · highClaude Code868885863−3±0Pass
2Claude Opus 5.5Anthropic · highClaude Code8581758110−2+1Pass
3Claude Sonnet 5.5Anthropic · highClaude Code7377867713+1−1Pass
4GPT-6 AstraOpenAI · highCodex5863686310−1+5Pass
4Qwen3.8 MaxAlibaba · mediumOpenCode6353786325––Pass
6GPT-6.1 SolOpenAI · highCodex585860582−2±0Pass
7Claude Haiku 5.5Anthropic · highClaude Code5343915348±0+6Pass
8Gemini 4 ArgonGoogle · mediumAntigravity5259485211−3−3Pass
9Gemini 4 HydrogenGoogle · xhighAntigravity4148524811+3–Pass
10Grok 4.7xAI · mediumCursor372930308−14−8Pass

Spread is the gap between the highest and the lowest judge. The last two columns show how the median moved since the earlier panels; “–” means the video was not in that panel yet. Differences of up to about 6 points are within the panel’s noise. Failed runs, such as a placeholder video, are left out even when the panel judged them.

Finished runs

Every run that finished with a valid video passing H1–H6. Model time counts only the time the model was generating; the rest of the wall time went to tools, renders and waiting. Select a model for its full record.

Select a column header to sort.

Key statistics of the finished runs
ModelEffortStatusScoreCostWall timeModel timetok/sTokensOutputCache hitH1–H6
Claude Fable 5.2Claude Code · AnthropichighFinished86$43.541 h 40 min1 h 8 min97.425.2M397k91%Pass
Claude Opus 5.5Claude Code · AnthropichighFinished81$15.411 h 7 min56 min105.323.2M352k97%Pass
Claude Sonnet 5.5Claude Code · AnthropichighFinished77$7.2853 min39 min133.019.5M307k95%Pass
GPT-6 AstraCodex · OpenAIhighFinished63$4.4614 min11 min36.22.6M25k97%Pass
Qwen3.8 MaxOpenCode · OpenRoutermediumFinished63$18.74 †4 h 6 min3 h 59 min30.969.2M443k99%Pass
GPT-6.1 SolCodex · OpenAIhighFinished58$0.8314 min13 min36.14.0M27k98%Pass
Claude Haiku 5.5Claude Code · AnthropichighFinished53$1.3229 min22 min214.712.3M289k96%Pass
Gemini 4 ArgonAntigravity · GooglemediumFinished52$3.5737 min27 min67.57.6M110k88%Pass
Gemini 4 HydrogenAntigravity · GooglexhighFinished · harness error, no report48$3.0120 min14 min190.66.9M162k87%Pass
Grok 4.7Cursor · CursormediumFinished30n/a31 min–––––Pass
Claude Opus 5.5 (run 2, older spec)Claude Code · Anthropic · older specdefaultFinished–$24.212 h47 min98.026.5M278k89%Pass
Nemotron 3 Ultra FreeOpenCode · OpenRoutermediumFinished–$0 †58 min27 min70.124.7M111k98%Pass
Nemotron 3.5 Super VL FreeOpenCode · OpenRouterthinkingFinished–$0 †58 min34 min101.830.1M205k98%Pass

† billed amount, not list price

8 runs that did not end with a valid video are not listed on this page. Their record is under Failures.

Score versus cost

Price and quality pull in opposite directions. A run is on the Pareto frontier when no other run is both cheaper and better.

Score versus cost (log scale)

  • Pareto frontier
Score versus costScatter plot of the latest panel score against the cost of each scored run, cost on a logarithmic axis, with the Pareto frontier drawn as a line.020406080100$0.5$1$2$5$10$20$50Cost (USD)Panel scoreClaude Fable 5.2: 86 · $43.54Fable 5.2Claude Opus 5.5: 81 · $15.41Opus 5.5Claude Sonnet 5.5: 77 · $7.28Sonnet 5.5GPT-6 Astra: 63 · $4.46AstraQwen3.8 Max: 63 · $18.74 †Qwen3.8 Max†GPT-6.1 Sol: 58 · $0.83SolClaude Haiku 5.5: 53 · $1.32Haiku 5.5Gemini 4 Argon: 52 · $3.57ArgonGemini 4 Hydrogen: 48 · $3.01Hydrogen
Each dot is a scored run; the red line joins the Pareto frontier. Costs are API list prices unless marked †. Free models sit in the band on the left. Runs without cost data are left out.

Cost per score point

  • Pareto frontier
  1. GPT-6.1 Sol$0.01458 points
  2. Claude Haiku 5.5$0.02553 points
  3. Gemini 4 Hydrogen$0.06348 points
  4. Gemini 4 Argon$0.06952 points
  5. GPT-6 Astra$0.07163 points
  6. Claude Sonnet 5.5$0.09577 points
  7. Claude Opus 5.5$0.19081 points
  8. Qwen3.8 Max$0.298†63 points
  9. Claude Fable 5.2$0.50686 points
Cost divided by the panel score. Lower is better. Free models cost nothing per point.
Value metrics of the scored runs
ModelScoreCostCost per pointPoints per $Output tokens per pointPareto frontier
Claude Fable 5.286$43.54$0.5062.04,614Cost · Wall time
Claude Opus 5.581$15.41$0.1905.34,344Cost · Wall time
Claude Sonnet 5.577$7.28$0.09510.63,989Cost · Wall time
GPT-6 Astra63$4.46$0.07114.1389Cost · Wall time
Qwen3.8 Max63$18.74 †$0.2983.47,036–
GPT-6.1 Sol58$0.83$0.01469.9473Cost
Claude Haiku 5.553$1.32$0.02540.25,446–
Gemini 4 Argon52$3.57$0.06914.62,121–
Gemini 4 Hydrogen48$3.01$0.06316.03,378–
Grok 4.730n/a––––

Time

Wall time runs from the agent’s first to its last action and includes rendering and encoding, which are part of the task.

Score versus wall time (log scale)

  • Pareto frontier
Score versus wall timeScatter plot of the latest panel score against the wall time of each scored run, time on a logarithmic axis, with the Pareto frontier drawn as a line.020406080100102050100200500Wall time (minutes)Panel scoreClaude Fable 5.2: 86 · 1 h 40 minFable 5.2Claude Opus 5.5: 81 · 1 h 7 minOpus 5.5Claude Sonnet 5.5: 77 · 53 minSonnet 5.5GPT-6 Astra: 63 · 14 minAstraQwen3.8 Max: 63 · 4 h 6 minQwen3.8 MaxGPT-6.1 Sol: 58 · 14 minSolClaude Haiku 5.5: 53 · 29 minHaiku 5.5Gemini 4 Argon: 52 · 37 minArgonGemini 4 Hydrogen: 48 · 20 minHydrogenGrok 4.7: 30 · 31 minGrok 4.7
Each dot is a scored run; the red line joins the runs no other run beats on both time and score.

Where the time went

  • Model generating
  • Tools & waiting
  1. Qwen3.8 Max4 h 6 min97% model
  2. Claude Opus 5.5 (run 2, older spec)2 h40% model
  3. Claude Fable 5.21 h 40 min68% model
  4. Claude Opus 5.51 h 7 min83% model
  5. Nemotron 3 Ultra Free58 min46% model
  6. Nemotron 3.5 Super VL Free58 min58% model
  7. Claude Sonnet 5.553 min73% model
  8. Gemini 4 Argon37 min74% model
  9. Claude Haiku 5.529 min78% model
  10. Gemini 4 Hydrogen20 min71% model
  11. GPT-6 Astra14 min81% model
  12. GPT-6.1 Sol14 min91% model
Model time is the sum of all model calls, from the request to the last token. Everything else is tool execution, renders, encodes and harness overhead. Runs without per-call timing data are left out.

Generation speed

Tokens per second while the model was actually generating. Tool calls and waits are excluded; time to the first token is included, so this is end-to-end model speed, not the pure decode rate.

Generated tokens per second

  1. Claude Haiku 5.5214.7 tok/sClaude Code
  2. Gemini 4 Hydrogen190.6 tok/sAntigravity
  3. Claude Sonnet 5.5133.0 tok/sClaude Code
  4. Claude Opus 5.5105.3 tok/sClaude Code
  5. Nemotron 3.5 Super VL Free101.8 tok/sOpenCode
  6. Claude Opus 5.5 (run 2, older spec)98.0 tok/sClaude Code
  7. Claude Fable 5.297.4 tok/sClaude Code
  8. Nemotron 3 Ultra Free70.1 tok/sOpenCode
  9. Gemini 4 Argon67.5 tok/sAntigravity
  10. GPT-6 Astra36.2 tok/sCodex
  11. GPT-6.1 Sol36.1 tok/sCodex
  12. Qwen3.8 Max30.9 tok/sOpenCode
Output and reasoning tokens divided by the summed generation time of all calls.

Tokens

Agents re-send their growing context on every call, so most tokens are cache reads. The mix differs a lot between harnesses: some write the cache explicitly, others only read it.

Token mix and cache-hit ratio

  • Uncached input
  • Cache read
  • Cache write
  • Output
  1. Qwen3.8 Max99% cache hit69.2M total
  2. Nemotron 3.5 Super VL Free98% cache hit30.1M total
  3. Claude Opus 5.5 (run 2, older spec)89% cache hit26.5M total
  4. Claude Fable 5.291% cache hit25.2M total
  5. Nemotron 3 Ultra Free98% cache hit24.7M total
  6. Claude Opus 5.597% cache hit23.2M total
  7. Claude Sonnet 5.595% cache hit19.5M total
  8. Claude Haiku 5.596% cache hit12.3M total
  9. Gemini 4 Argon88% cache hit7.6M total
  10. Gemini 4 Hydrogen87% cache hit6.9M total
  11. GPT-6.1 Sol98% cache hit4.0M total
  12. GPT-6 Astra97% cache hit2.6M total
Share of each token type in the run’s total. Cache hit = cache reads divided by all input tokens (uncached + cache read + cache write). Output includes reasoning.

Judges

The panel score hides how the judges arrived at it. These charts show what they rewarded, how much they agreed, and how stable the scores were across three separate panels.

What earned the points

  • Landmarks
  • City look
  • Camera & drone
  • Life
  • Overall
  1. Claude Fable 5.286Median
  2. Claude Opus 5.581Median
  3. Claude Sonnet 5.577Median
  4. GPT-6 Astra63Median
  5. Qwen3.8 Max63Median
  6. GPT-6.1 Sol58Median
  7. Claude Haiku 5.553Median
  8. Gemini 4 Argon52Median
  9. Gemini 4 Hydrogen48Median
  10. Grok 4.730Median
Mean of the three judges per rubric category in the latest panel (landmarks 25, city look 25, camera & drone 30, life 10, overall 10). The mean of the parts can differ slightly from the median score.

Judge agreement per video

  • Anthropic
  • OpenAI
  • Google
  • Median
Judge agreement per videoDot plot with one row per video showing each judge’s total score, the range between them, and the median.0255075100Claude Fable 5.2: Anthropic 86, OpenAI 88, Google 85; Median 86Fable 5.2Δ 3Claude Opus 5.5: Anthropic 85, OpenAI 81, Google 75; Median 81Opus 5.5Δ 10Claude Sonnet 5.5: Anthropic 73, OpenAI 77, Google 86; Median 77Sonnet 5.5Δ 13GPT-6 Astra: Anthropic 58, OpenAI 63, Google 68; Median 63AstraΔ 10Qwen3.8 Max: Anthropic 63, OpenAI 53, Google 78; Median 63Qwen3.8 MaxΔ 25GPT-6.1 Sol: Anthropic 58, OpenAI 58, Google 60; Median 58SolΔ 2Claude Haiku 5.5: Anthropic 53, OpenAI 43, Google 91; Median 53Haiku 5.5Δ 48Gemini 4 Argon: Anthropic 52, OpenAI 59, Google 48; Median 52ArgonΔ 11Gemini 4 Hydrogen: Anthropic 41, OpenAI 48, Google 52; Median 48HydrogenΔ 11Grok 4.7: Anthropic 37, OpenAI 29, Google 30; Median 30Grok 4.7Δ 8
One row per video in the latest panel. The bar spans the lowest to the highest judge; the vertical tick is the median.

Score drift across panels

  • moved more than 6 points
  • stable within 6 points
Score drift across panelsSlope chart of each model’s median score in the three blind panels.02040608010011 Nov 202613 Nov 202614 Nov 2026Claude Fable 5.2: 11 Nov 86, 13 Nov 89, 14 Nov 86Fable 5.2 86Claude Opus 5.5: 11 Nov 80, 13 Nov 83, 14 Nov 81Opus 5.5 81Claude Sonnet 5.5: 11 Nov 78, 13 Nov 76, 14 Nov 77Sonnet 5.5 77GPT-6 Astra: 11 Nov 58, 13 Nov 64, 14 Nov 63Astra 63Qwen3.8 Max: 11 Nov –, 13 Nov –, 14 Nov 63Qwen3.8 Max 63GPT-6.1 Sol: 11 Nov 58, 13 Nov 60, 14 Nov 58Sol 58Claude Haiku 5.5: 11 Nov 47, 13 Nov 53, 14 Nov 53Haiku 5.5 53Gemini 4 Argon: 11 Nov 55, 13 Nov 55, 14 Nov 52Argon 52Gemini 4 Hydrogen: 11 Nov –, 13 Nov 45, 14 Nov 48Hydrogen 48Grok 4.7: 11 Nov 38, 13 Nov 44, 14 Nov 30Grok 4.7 30
The same videos, re-judged blind with fresh letters each time a new model joined. Red lines moved by more than 6 points between two panels.
Rank stability between consecutive panels
PanelsVideos in bothSpearman ρLargest rank changeMean |Δ score|Largest |Δ score|
11 Nov → 13 Nov80.99413.56
13 Nov → 14 Nov90.98313.214

By harness and provider

The model is never measured alone: it runs inside a harness, and the provider decides prices and limits.

Runs grouped by harness and provider
HarnessProviderRunsScoredBest scoreTotal costTotal time
Claude CodeAnthropic54Claude Fable 5.2 86$91.766 h 8 min
OpenCodeOpenRouter31Qwen3.8 Max 63$18.746 h 2 min
AntigravityGoogle22Gemini 4 Argon 52$6.5857 min
CodexOpenAI22GPT-6 Astra 63$5.2928 min
CursorCursor11Grok 4.7 30n/a31 min