GAUGE · Selection
Selection.
Pick the relevant items from a list.
select.sections
Pick the document sections (by title) that answer a question, from a list of section titles.
Best score
gpt-6.1-sol · minimum thinking
93.8 ±5.2
Tied with the best
1 model
within 5.2 of the best
Fastest among tied
gpt-6.1-sol · minimum thinking
median · API round trip
Cheapest API among tied
gpt-6.1-sol · minimum thinking
$2.0 per 1,000 calls
Best local model
gemma4:31b-mlx
72.0 ±8.7
| Model | Type | Score ± 95% | n | Repeats | Consistency | Validity | Median latency | $ / 1k calls |
|---|---|---|---|---|---|---|---|---|
| gpt-6.1-sol · minimum thinking | API | 93.8 ±5.2 | 100 | 3 | — | 100.0% | 1.4 s API | $2.0 |
| gemini-4-argon · minimum thinking | API | 83.8 ±7.3 | 100 | 3 | — | 100.0% | 8.0 s API | $5.1 |
| gpt-6.1-terra | API | 82.7 ±7.5 | 100 | 3 | — | 100.0% | 1.1 s API | $0.94 |
| gemini-3.8-flash | API | 81.5 ±7.6 | 100 | 3 | — | 100.0% | 2.1 s API | $1.7 |
| claude-opus-5-5 · minimum thinking | API | 80.5 ±7.8 | 100 | 3 | — | 100.0% | 1.6 s API | $10.5 |
| claude-sonnet-5-5 | API | 78.7 ±8.0 | 100 | 3 | — | 100.0% | 1.6 s API | $5.3 |
| gpt-6-luna | API | 73.0 ±8.6 | 100 | 3 | — | 100.0% | 835 ms API | $0.10 |
| gemma4:31b-mlx · 31.7B | Local | 72.0 ±8.7 | 100 | 1 | — | 100.0% | 1.7 s local | — |
| gpt-5.4 | API | 69.8 ±8.9 | 100 | 3 | — | 100.0% | 849 ms API | $1.7 |
| claude-haiku-5-5 | API | 68.0 ±9.0 | 100 | 3 | — | 100.0% | 913 ms API | $0.27 |
| gemma4:12b-mlx · 12.4B | Local | 67.5 ±9.0 | 100 | 1 | — | 100.0% | 727 ms local | — |
| qwen3.8:27b-mlx · 27.8B | Local | 63.5 ±9.3 | 100 | 1 | — | 100.0% | 1.5 s local | — |
| bonsai-2:27b · 26.9B | Local | 55.0 ±9.6 | 100 | 1 | — | 100.0% | 3.7 s local | — |
| ministral-3:14b · 13.9B | Local | 46.0 ±9.6 | 100 | 1 | — | 100.0% | 1.7 s local | — |
Amber edge and bar: tied with the best (score ≥ best − its 95% margin, harness rule). Only rows with the task's full n are compared.
Smaller sample · n = 20 · not compared
| Model | Type | Score ± 95% | n | Repeats | Consistency | Validity | Median latency | $ / 1k calls |
|---|---|---|---|---|---|---|---|---|
| gpt-6.1-sol · thinking on | API | 93.3 ±13.9 | 20 | 3 | — | 100.0% | 2.9 s API | $2.9 |
| gpt-5.4 · thinking on | API | 87.5 ±15.6 | 20 | 3 | — | 100.0% | 6.5 s API | $16.3 |
| gpt-6.1-terra · thinking on | API | 85.8 ±16.1 | 20 | 3 | — | 100.0% | 2.1 s API | $1.4 |
| gpt-6-luna · thinking on | API | 84.2 ±16.4 | 20 | 3 | — | 100.0% | 2.9 s API | $0.24 |
| gemini-3.8-flash · thinking on | API | 80.8 ±17.2 | 20 | 3 | — | 100.0% | 24.3 s API | $18.7 |
| claude-sonnet-5-5 · thinking on | API | 80.0 ±17.3 | 20 | 3 | — | 100.0% | 1.1 s API | $5.6 |
| gemini-4-argon · thinking on | API | 80.0 ±17.3 | 20 | 3 | — | 100.0% | 15.9 s API | $12.4 |
| claude-opus-5-5 · thinking on | API | 76.7 ±17.9 | 20 | 3 | — | 100.0% | 1.5 s API | $11.5 |
| claude-haiku-5-5 · thinking on | API | 75.8 ±18.1 | 20 | 3 | — | 100.0% | 1.8 s API | $0.41 |
Measured on fewer cases than the task's n = 100. These rows are outside the winner comparison above and have wider margins; read them on their own.