Coding agent benchmarks
Codex Radar runs a shared set of real coding tasks through each model and effort and reports IQ, cost and minutes. Leaderboard rows below keep the harness each score was measured with.
Codex Radar runs · codex
| Model | Effort | Radar IQ | Avg cost (USD) | Avg minutes |
|---|---|---|---|---|
| Claude Sonnet 5 | high | 150.0 | $4.51 | 20.9 |
| Kimi K2.8 Preview | high | 150.0 | — | 14.7 |
| Kimi K2.8 Preview | max | 150.0 | — | 16.0 |
| Claude Opus 5 5 | low | 150.0 | $0.554 | 3.5 |
| Claude Opus 5 5 | medium | 150.0 | $0.665 | 3.3 |
| Claude Opus 5 5 | xhigh | 150.0 | $2.61 | 11.0 |
| Claude Opus 5 5 | max | 150.0 | $22 | 84.3 |
| Claude Opus 5 | medium | 112.5 | $5.01 | 15.1 |
| GPT-6 Astra | high | 110.4 | $2.81 | 13.8 |
| GPT-6 Astra | ultra | 110.2 | $8.75 | 15.1 |
| GPT 5.6 Sol | max | 108.5 | $6.34 | 33.9 |
| Grok 4.7 | xhigh | 108.3 | $13 | 47.0 |
| GPT-6 Astra | max | 108.0 | $4.64 | 23.3 |
| GPT-6 Astra | medium | 107.5 | $1.83 | 9.0 |
| Grok 4.6 | xhigh | 107.1 | $3.25 | 34.3 |
| GPT-6 Astra | xhigh | 105.3 | $3.60 | 17.9 |
| Grok 4.7 | medium | 105.3 | $5.92 | 35.4 |
| Grok 4.7 | high | 105.1 | $9.36 | 42.4 |
| GPT 5.6 Luna | max | 102.2 | $0.426 | 37.6 |
| Grok 4.7 | low | 101.3 | $2.76 | 19.1 |
| GPT 5.6 Terra | max | 100.9 | $2.80 | 30.9 |
| K3 | max | 100.8 | $3.33 | 59.1 |
| GPT 5.6 Sol | ultra | 100.5 | $18 | 51.9 |
| Claude Opus 5 | low | 100.0 | $1.42 | 10.7 |
| Grok 4.6 | high | 99.5 | $2.71 | 30.2 |
| Hy4 Preview | high | 99.5 | $0.656 | 38.8 |
| GPT 5.6 Terra | ultra | 99.1 | $7.65 | 41.9 |
| GPT-6 Sol | high | 98.7 | $0.765 | 11.4 |
| GPT-6 Astra | low | 98.4 | $1.53 | 8.6 |
| GPT 5.6 Sol | xhigh | 98.2 | $4.09 | 24.4 |
| GPT-6 Sol | max | 97.4 | $1.27 | 16.4 |
| GPT 5.5 | xhigh | 96.9 | $4.76 | 22.5 |
| GLM 5.3 (v3) | high | 96.7 | $1.61 | 33.8 |
| GPT 5.6 Sol | high | 96.4 | $3.31 | 19.9 |
| GLM 5.3 Flash (v3) | max | 95.9 | $0.298 | 49.2 |
| GPT-6 Sol | ultra | 95.9 | $2.52 | 16.1 |
| GPT 5.6 Sol | medium | 95.5 | $2.26 | 16.0 |
| Grok 4.6 | medium | 95.1 | $2.25 | 23.9 |
| Hy4 Preview | low | 95.1 | $0.651 | 35.9 |
| Claude Opus 5 | xhigh | 94.2 | $6.36 | 19.0 |
| K3 | high | 94.0 | $1.92 | 33.4 |
| Gemini 3.7 Flash | high | 94.0 | $2.00 | 15.1 |
| GLM 5.3 (v3) | max | 93.9 | $1.86 | 35.5 |
| GLM 5.3 Flash (v3) | high | 92.3 | $0.325 | 48.7 |
| GPT-6 Sol | xhigh | 92.3 | $0.941 | 13.0 |
| Gemini 3.8 Flash | medium | 92.1 | $2.13 | 16.2 |
| Gemini 3.8 Flash | high | 90.0 | $2.64 | 22.9 |
| GPT 5.5 | high | 89.7 | $3.12 | 16.2 |
| Gemini 3.7 Flash | medium | 89.5 | $1.69 | 14.4 |
| DeepSeek V4.1 Flash | high | 89.4 | $0.109 | 18.4 |
| K3 | low | 88.0 | $0.931 | 22.8 |
| GPT 5.6 Terra | xhigh | 88.0 | $1.53 | 19.3 |
| DeepSeek V4.1 Flash | max | 87.5 | $0.146 | 18.9 |
| Claude Opus 5 | max | 87.5 | $13 | 32.1 |
| GPT-6 Sol | medium | 86.3 | $0.541 | 9.2 |
| GPT 5.6 Luna | xhigh | 86.2 | $0.256 | 24.6 |
| DeepSeek V4 Flash | max | 85.7 | $0.211 | 31.3 |
| GPT 5.6 Sol | low | 81.3 | $1.16 | 10.4 |
| GLM 5.3 (v3) | low | 80.3 | $0.622 | 15.0 |
| GPT-6 Sol | low | 80.3 | $0.317 | 5.8 |
| GPT-6 Luna | max | 80.0 | $0.067 | 22.5 |
| Grok 4.6 | low | 78.6 | $0.989 | 11.9 |
| GPT 5.6 Terra | high | 77.2 | $0.898 | 13.6 |
| GPT 5.6 Luna | high | 76.3 | $0.162 | 17.3 |
| Gemini 3.7 Flash | low | 75.6 | $1.67 | 14.0 |
| Claude Sonnet 5 | medium | 75.0 | $1.93 | 9.8 |
| Claude Opus 5 | high | 75.0 | $5.73 | 17.3 |
| Hy4 Preview | max | 74.2 | $0.628 | 58.8 |
| GLM 5.3 Flash (v3) | low | 70.5 | $0.094 | 24.4 |
| Gemini 3.8 Flash | low | 67.5 | $3.09 | 17.4 |
| GPT-6 Luna | xhigh | 65.7 | $0.067 | 23.7 |
| DeepSeek V4 Flash | high | 65.6 | $0.187 | 30.4 |
| Kimi K2.8 Preview | low | 64.0 | — | 18.3 |
| Claude Sonnet 5 | low | 60.9 | $0.895 | 13.9 |
| GPT 5.6 Terra | medium | 59.8 | $0.547 | 9.4 |
| GPT 5.6 Terra | low | 50.9 | $0.439 | 8.0 |
| Claude Sonnet 5 | xhigh | 42.9 | $5.45 | 25.0 |
| GPT-6 Luna | high | 41.9 | $0.023 | 8.0 |
| GPT 5.6 Luna | medium | 32.6 | $0.066 | 10.7 |
| GPT-6 Luna | medium | 22.4 | $0.011 | 6.6 |
| GPT-6 Luna | low | 8.5 | $0.029 | 12.0 |
| GPT 5.6 Luna | low | 7.1 | $0.026 | 7.1 |
| Claude Sonnet 5 | max | 0.0 | $8.19 | 39.6 |
50530 runs recorded in total. Oct 1, 2026
Codex Radar runs · dsh
| Model | Effort | Radar IQ | Avg cost (USD) | Avg minutes |
|---|---|---|---|---|
| Dsh Deepseek V4.1 Flash | high | 101.2 | $0.185 | 45.7 |
| Dsh Deepseek V4.1 Flash | max | 99.3 | $0.220 | 36.6 |
| Dsh Deepseek V4 Flash Vision Exp | max | 97.8 | $0.259 | 30.1 |
| Dsh Deepseek V4 Flash | max | 80.2 | $0.169 | 31.4 |
| Dsh Deepseek V4 Flash | high | 75.0 | $0.154 | 26.5 |
| Dsh Deepseek V4 Flash Vision Exp | high | 68.8 | $0.154 | 20.0 |
50530 runs recorded in total. Oct 1, 2026
Community ratings
| Model | Average / 10 | Ratings |
|---|---|---|
| GPT-6 Astra high | 10.0 | 1 |
| GPT-5.6 Sol medium | 10.0 | 1 |
| GPT-5.6 Terra ultra | 10.0 | 1 |
| GPT-5.6 Luna low | 10.0 | 1 |
| GPT-6 Astra xhigh | 9.0 | 2 |
| GPT-6 Astra ultra | 8.7 | 3 |
| GPT-6.1 Sol max | 8.6 | 7 |
| GPT-6.1 Sol high | 8.2 | 9 |
| GPT-6 Astra medium | 8.0 | 1 |
| GPT-5.6 Sol xhigh | 8.0 | 1 |
| GPT-5.6 Luna max | 8.0 | 1 |
| GPT-6.1 Sol xhigh | 8.0 | 5 |
| GPT-6.1 Sol medium | 8.0 | 1 |
| GPT-6 Astra max | 7.0 | 2 |
| GPT-6.1 Sol low | 6.0 | 2 |
| GPT-6 Sol xhigh | 4.0 | 1 |
Oct 18, 2026
Terminal-Bench 4.0 Resolution Rate % · Sep 21, 2026
| Model | Harness | Effort | Score | Cost per task |
|---|---|---|---|---|
| Claude Opus 5.5 | Claude Code | — | 66.4 | — |
| GPT-6 Astra | Codex | max | 58.2 | — |
| Claude Fable 5.1 | Claude Code | max | 57.9 | — |
| GPT-6 Astra | Codex | xhigh | 57.9 | — |
| GPT-6 Astra | Codex | high | 57.9 | — |
| Claude Fable 5.1 | Claude Code | xhigh | 57.9 | — |
| Claude Fable 5.1 | Claude Code | high | 54.5 | — |
| GPT-6 Astra | Codex | medium | 54.2 | — |
| Claude Fable 5.1 | Claude Code | medium | 53.9 | — |
| Claude Opus 5 | Claude Code | xhigh | 53.9 | — |
| Claude Opus 5 | Claude Code | max | 51.8 | — |
| GPT-6 Astra | Codex | low | 50.6 | — |
| Claude Opus 5 | Claude Code | high | 50.3 | — |
| Claude Opus 5 | Claude Code | medium | 44.9 | — |
| Claude Fable 5 | Claude Code | max | 44.5 | — |
| Claude Fable 5.1 | Claude Code | low | 43.3 | — |
| GLM 5.3 (v3) | Claude Code | max | 41.8 | — |
| Grok 4.7 | Grok Build | xhigh | 37.6 | — |
| GPT 5.6 Sol | Codex | max | 37.3 | — |
| Claude Opus 5 | Claude Code | low | 34.9 | — |
| DeepSeek V4.1 Flash | DeepSeek Harness Minimal | max | 31.2 | — |
| SWE-2 | Devin CLI | — | 27.3 | — |
| Claude Opus 4.8 | Claude Code | max | 23.6 | — |
| GPT 5.6 Terra | Codex | max | 21.5 | — |
| Grok 4.6 | Grok Build | high | 20.3 | — |
Artificial Analysis Coding Agent Index v1.5 Coding Agent Index · Oct 1, 2026
| Model | Harness | Effort | Score | Cost per task |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Claude Code | max | 68.4 | $14 |
| Claude Opus 5.5 | Claude Code | max | 66.0 | $13 |
| GPT-6.1 Sol | Codex | xhigh | 62.9 | $1.04 |
| Claude Sonnet 5.5 | Claude Code | xhigh | 62.9 | $3.33 |
| Claude Fable 5.1 | Claude Code | max | 62.2 | $12 |
| Claude Fable 5.1 | Devin Fusion CLI | xhigh | 61.7 | $7.90 |
| GPT-6 Astra | Codex | max | 61.6 | $7.47 |
| GPT-6.1 Sol | Codex | medium | 61.4 | $0.705 |
| GPT-6.1 Sol | Codex | high | 60.1 | $0.889 |
| GPT-6.1 Sol | Codex | max | 60.1 | $1.55 |
| Claude Opus 5 | Claude Code | max | 59.7 | $11 |
| GPT-6 Astra | Devin Fusion CLI | xhigh | 58.9 | $4.54 |
| GPT-6.1 Sol | Codex | low | 57.2 | $0.499 |
| GPT-6 Sol | Codex | max | 56.7 | $2.99 |
| Grok 4.7 | Grok Build | xhigh | 56.3 | $8.82 |
| Claude Sonnet 5.5 | Claude Code | high | 55.0 | $1.24 |
| GPT 5.6 Sol | Codex | max | 54.6 | $6.35 |
| Muse Spark 1.3 | Muse Code | max | 54.3 | $3.98 |
| GLM 5.3 (v3) | Opencode | max | 53.6 | $4.24 |
| Kimi K3 (v1) | Kimi Code CLI | — | 51.9 | $5.05 |
| Muse Spark 1.3 | Muse Code 1.0.2 RC | xhigh | 48.3 | $3.47 |
| Grok 4.6 | Grok Build | xhigh | 47.0 | $3.57 |
| Claude Sonnet 5.5 | Claude Code | medium | 45.9 | $0.619 |
| Qwen3.8 Max | Claude Code | max | 43.3 | $3.48 |
| GPT 5.6 Luna | Codex | max | 43.2 | $0.438 |
DeepSWE v1.1 Pass@1 % · Sep 3, 2026
| Model | Harness | Effort | Score | Cost per task |
|---|---|---|---|---|
| DeepSeek V4.1 Flash | mini-SWE | max | 74.2 | — |
| GPT-6 Astra | mini-swe-agent | xhigh | 74.1 | $6.52 |
| Gemini 3.8 Flash | mini-swe-agent | high | 73.8 | $2.36 |
| Claude Opus 5 | mini-swe-agent | max | 73.6 | $12 |
| GPT-6 Astra | mini-swe-agent | high | 73.2 | $5.72 |
| GPT-6 Astra | mini-swe-agent | max | 73.2 | $12 |
| Claude Opus 5 | mini-swe-agent | xhigh | 73.2 | $9.07 |
| Claude Opus 5 | mini-swe-agent | high | 72.8 | $6.08 |
| GPT-6 Astra | mini-swe-agent | medium | 72.8 | $4.38 |
| GPT 5.6 Sol | mini-swe-agent | max | 72.7 | $8.39 |
| MiMo V2.6 Pro | mini-swe-agent | — | 71.9 | — |
| Gemini 3.8 Flash | mini-swe-agent | medium | 71.0 | $1.97 |
| Grok 4.7 | mini-swe-agent | high | 71.0 | — |
| GPT 5.6 Sol | mini-swe-agent | xhigh | 70.7 | $4.70 |
| Claude Fable 5 | mini-swe-agent | xhigh | 69.9 | $13 |
| Claude Fable 5 | mini-swe-agent | max | 69.7 | $22 |
| GPT 5.6 Terra | mini-swe-agent | max | 69.6 | $4.95 |
| GPT 5.6 Sol | mini-swe-agent | high | 69.4 | $3.47 |
| GLM 5.3 (v3) | mini-swe-agent | max | 69.0 | $3.99 |
| Claude Opus 5 | mini-swe-agent | medium | 68.9 | $3.29 |
| Claude Fable 5 | mini-swe-agent | high | 68.6 | $9.18 |
| Kimi K3 (v1) | mini-swe-agent | max | 68.5 | $4.65 |
| MiMo V2.6 Flash | mini-swe-agent | — | 67.9 | — |
| Grok 4.6 | mini-swe-agent | medium | 67.5 | $3.45 |
| GPT 5.6 Luna | mini-swe-agent | max | 67.2 | $3.03 |