Stats.
Runtime, tokens, and turns across models and coding agents.
1818 latest finished attempts · 41 not published · Median per run · Available coverage; not matched across models
Model overview
Click a model to inspect the underlying runs.
| Model / agent | Finished runs | Not published | ||||
|---|---|---|---|---|---|---|
| Muse Spark OpenPi · Lunaroute | 1m 35s128 measured | 321.8K128 measured | 14128 measured | 16128 measured | 128 | 0 |
| Muse Wonder 1.0Pi · Lunaroute | 1m 44s24 measured | 164K24 measured | 1024 measured | 9.524 measured | 24 | 8 |
| BeamPi · Lunaroute | 2m 21s76 measured | 388.7K76 measured | 2276 measured | 2276 measured | 76 | 10 |
| GPT-6 LunaCodex | 2m 48s111 measured | 335.7K111 measured | 12111 measured | 11111 measured | 111 | 1 |
| Claude Sonnet 5.5Claude Code | 3m 23s128 measured | 379.2K128 measured | 12128 measured | 16.5128 measured | 128 | 0 |
| Claude Haiku 5.5Claude Code | 3m 55s51 measured | 434.6K51 measured | 1151 measured | 1651 measured | 51 | 1 |
| DeepSeek 4.1 Flash · BasePi · Lunaroute | 4m 26s128 measured | 1.5M128 measured | 31128 measured | 37128 measured | 128 | 0 |
| Muse Spark 1.3Pi · Lunaroute | 4m 36s128 measured | 352.3K128 measured | 16128 measured | 16128 measured | 128 | 0 |
| GLM 5.3 FlashPi · Lunaroute | 4m 39s128 measured | 988.4K128 measured | 28.5128 measured | 33128 measured | 128 | 0 |
| GPT-6 SolCodex | 5m 34s111 measured | 536K111 measured | 16111 measured | 15111 measured | 111 | 1 |
| GPT-5.6 SolCodex | 5m 46s111 measured | 664.4K111 measured | 19111 measured | 18111 measured | 111 | 3 |
| GLM 5.2Pi · Lunaroute | 6m 53s128 measured | 845.2K128 measured | 29128 measured | 30128 measured | 128 | 9 |
| Claude Opus 5.5Claude Code | 7m 44s41 measured | 622.3K41 measured | 1441 measured | 2641 measured | 41 | 2 |
| DeepSeek 4.1 Flash · 1017Pi · Lunaroute | 9m 01s128 measured | 3.5M128 measured | 48.5128 measured | 54128 measured | 128 | 0 |
| Claude Sonnet 5Claude Code | 10m 12s128 measured | 2.2M128 measured | 34128 measured | 43128 measured | 128 | 0 |
| GPT-6 AstraCodex | 10m 31s107 measured | 408.4K107 measured | 12107 measured | 11107 measured | 107 | 2 |
| GLM 5.3Pi · Lunaroute | 12m 15s128 measured | 1.4M128 measured | 32128 measured | 37128 measured | 128 | 0 |
| Claude Opus 5Claude Code | 17m 40s34 measured | 4.5M34 measured | 5534 measured | 61.534 measured | 34 | 4 |
Counts matter: models with different coverage are not a matched comparison. Use Shared cells for the same brief × version × skill × sample across every displayed model.
Model × skill
Each cell aggregates the selected briefs and samples.
How these numbers are calculated
Generation time is the recorded generation duration, including agent setup where the runner includes it. It excludes screenshot capture, evaluation, and priority queue wait. Total elapsed time includes capture and evaluation. Missing measurements appear as a dash, never an inferred zero.
Reported tokens uses the runner’s recorded total, summed over its responses. It includes reused context, not just newly written code. Codex input includes cached input; Claude and Pi expose cache read/write separately. We do not add cache counters to reported totals again. Tokenizers and accounting differ; this is usage, not a price comparison. Output tokens are available in the matrix and run details. Zero means the runner recorded zero, which can happen before a provider error.
Turns means assistant responses in Pi, completed model responses in Codex, and unique assistant messages/native turn counts in Claude Code. A tool call is separate, and agents can make several per turn. Tool calls and turns are not interchangeable across agents.
Latest per cell chooses the latest attempt before filtering outcomes. Unpublished attempts are included by default and counted separately. Partial evaluator results remain included under “Completed generation.” All attempts includes retries. Median/mean give each included run equal weight; totals reflect the selected coverage. Live and queued runs are excluded from aggregates. No design quality ranking is implied.