GAUGE · Cross-checking
Cross-checking.
Decide whether two documents match on a stated criterion.
match.legal_case
Decide whether a long legal document addresses the same or a directly analogous offence as a case file (yes/no plus explanation).
Top of the tied group
2 models, not separated
highest point estimate 100.0 ±2.6, at n = 100
Tied with the best
2 models (3 rows)
within 2.6 of the best
Fastest among tied
gpt-6.1-terra
median · API round trip
Cheapest API among tied
gpt-6.1-terra
$32.3 per 1,000 calls
—
Best local model
None
none in the tied group
| Model | Type | Score ± 95% | n | Repeats | Consistency | Validity | Median latency | $ / 1k calls |
|---|---|---|---|---|---|---|---|---|
| gpt-6.1-terra | API | 100.0 ±2.6 | 100 | 1 | — | 100.0% | 2.3 s API | $32.3 |
| gpt-6.1-sol · minimum thinking | API | 99.0 ±3.2 | 100 | 1 | — | 100.0% | 3.2 s API | $65.0 |
| gpt-6.1-sol · thinking on | API | 98.0 ±3.7 | 100 | 1 | — | 100.0% | 7.3 s API | $67.6 |
| gpt-5.4 · thinking on | API | 97.0 ±4.1 | 100 | 1 | — | 100.0% | 10.4 s API | $94.3 |
| gpt-6.1-terra · thinking on | API | 97.0 ±4.1 | 100 | 1 | — | 100.0% | 4.6 s API | $33.2 |
| gemini-4-argon · minimum thinking | API | 96.0 ±4.5 | 100 | 1 | — | 100.0% | 8.0 s API | $71.6 |
| gpt-6-luna · thinking on | API | 96.0 ±4.5 | 100 | 1 | — | 100.0% | 3.7 s API | $3.4 |
| claude-opus-5-5 · thinking on | API | 95.0 ±4.8 | 100 | 1 | — | 100.0% | 3.7 s API | $193.7 |
| gemini-4-argon · thinking on | API | 95.0 ±4.8 | 100 | 1 | — | 100.0% | 19.9 s API | $81.7 |
| gemini-3.8-flash | API | 93.0 ±5.4 | 100 | 1 | — | 100.0% | 2.8 s API | $26.8 |
| gpt-5.4 | API | 93.0 ±5.4 | 100 | 1 | — | 100.0% | 3.4 s API | $85.4 |
| claude-opus-5-5 · minimum thinking | API | 90.0 ±6.1 | 100 | 1 | — | 100.0% | 2.9 s API | $191.4 |
| claude-sonnet-5-5 · thinking on | API | 89.0 ±6.4 | 100 | 1 | — | 100.0% | 2.6 s API | $96.1 |
| gemini-3.8-flash · thinking on | API | 87.0 ±6.8 | 100 | 1 | — | 100.0% | 10.3 s API | $31.2 |
| gpt-6-luna | API | 86.0 ±6.9 | 100 | 1 | — | 100.0% | 1.9 s API | $3.2 |
| claude-sonnet-5-5 | API | 84.0 ±7.3 | 100 | 1 | — | 100.0% | 2.6 s API | $95.3 |
| claude-haiku-5-5 · thinking on | API | 82.0 ±7.6 | 100 | 1 | — | 100.0% | 3.1 s API | $18.7 |
| claude-haiku-5-5 | API | 78.0 ±8.1 | 100 | 1 | — | 99.0% | 1.7 s API | $18.4 |
Amber edge and bar: tied with the best (score ≥ best − its 95% margin, harness rule). Only rows with the task's full n are compared.
Smaller sample · n = 20 · not compared
| Model | Type | Score ± 95% | n | Repeats | Consistency | Validity | Median latency | $ / 1k calls |
|---|---|---|---|---|---|---|---|---|
| bonsai-2:27b · 26.9B | Local | 85.0 ±16.3 | 20 | 1 | — | 100.0% | 35.6 s local | — |
| gemma4:31b-mlx · 31.7B | Local | 85.0 ±16.3 | 20 | 1 | — | 100.0% | 30.6 s local | — |
| qwen3.8:27b-mlx · 27.8B | Local | 85.0 ±16.3 | 20 | 1 | — | 100.0% | 24.2 s local | — |
| gemma4:12b-mlx · 12.4B | Local | 65.0 ±19.4 | 20 | 1 | — | 100.0% | 11.4 s local | — |
| ministral-3:14b · 13.9B | Local | 55.0 ±19.9 | 20 | 1 | — | 100.0% | 53.6 s local | — |
| pplx-decider:27b | Decision | 50.0 ±20.0 | 20 | 1 | — | 55.0% | 13.2 s local | — |
| d3:27b | Decision | 5.0 ±13.2 | 20 | 1 | — | 10.0% | 29.5 s local | — |
Measured on fewer cases than the task's n = 100. These rows are outside the winner comparison above and have wider margins; read them on their own.