GAUGE · Embeddings
Embeddings.
Retrieve the right items by vector similarity.
embed.example_retrieval
Retrieve labelled routing examples for a question (recall, and whether the majority label matches the route).
Top of the tied group
6 models, not separated
highest point estimate 38.2 ±6.7, at n = 201
Tied with the best
6 models
within 6.7 of the best
Cheapest API among tied
text-embedding-3-small
$0.00053 per 1,000 queries
Best local model
snowflake-arctic-embed2
38.2 ±6.7 · tied
| Model | Type | Score ± 95% | n | Label match | Embed time | $ / 1k queries |
|---|---|---|---|---|---|---|
| snowflake-arctic-embed2 | Local | 38.2 ±6.7 | 201 | 26.3% | 4.8 s | — |
| nomic-embed-text-v2-moe | Local | 37.3 ±6.6 | 201 | 30.0% | 4 s | — |
| text-embedding-3-large | API | 36.8 ±6.6 | 201 | 27.5% | 4.4 s | $0.0035 |
| text-embedding-3-small | API | 34.7 ±6.5 | 201 | 26.3% | 3 s | $0.00053 |
| qwen3-embedding:0.6b | Local | 34.6 ±6.5 | 201 | 28.6% | 6.4 s | — |
| qwen3-embedding:8b | Local | 32.9 ±6.5 | 201 | 29.9% | 26.9 s | — |
| granite-embedding:278m | Local | 30.3 ±6.3 | 201 | 26.3% | 3.7 s | — |
| bge-m3 | Local | 29.9 ±6.3 | 201 | 30.0% | 4.6 s | — |
| embeddinggemma-2:570m | Local | 29.4 ±6.3 | 201 | 26.3% | 6.1 s | — |
| embeddinggemma-2:740m | Local | 29.4 ±6.3 | 201 | 26.3% | 5.7 s | — |
| gemini-embedding-001 | API | 29.2 ±6.3 | 201 | 27.5% | 6.8 s | $0.0032 |
| gemini-embedding-2 | API | 28.7 ±6.2 | 201 | 26.3% | 7.7 s | $0.0058 |
| paraphrase-multilingual | Local | 26.9 ±6.1 | 201 | 26.3% | 3.9 s | — |
Amber edge and bar: tied with the best (score ≥ best − its 95% margin, harness rule). Only rows with the task's full n are compared.
embed.task_retrieval
Retrieve the labelled example emails the classifier is shown for an incoming email (embedding recall against the platform's top 5).
Top of the tied group
13 models, not separated
highest point estimate 24.8 ±8.4, at n = 100
Tied with the best
13 models
within 8.4 of the best
Cheapest API among tied
text-embedding-3-small
$0.011 per 1,000 queries
Best local model
nomic-embed-text-v2-moe
24.2 ±8.3 · tied
| Model | Type | Score ± 95% | n | Label match | Embed time | $ / 1k queries |
|---|---|---|---|---|---|---|
| text-embedding-3-large | API | 24.8 ±8.4 | 100 | — | 3.3 s | $0.072 |
| nomic-embed-text-v2-moe | Local | 24.2 ±8.3 | 100 | — | 3.3 s | — |
| bge-m3 | Local | 22.6 ±8.2 | 100 | — | 5.2 s | — |
| qwen3-embedding:8b | Local | 22.0 ±8.1 | 100 | — | 66 s | — |
| gemini-embedding-2 | API | 22.0 ±8.1 | 100 | — | 4.8 s | $0.084 |
| text-embedding-3-small | API | 21.2 ±8.0 | 100 | — | 3.1 s | $0.011 |
| embeddinggemma-2:570m | Local | 21.0 ±8.0 | 100 | — | 5.9 s | — |
| embeddinggemma-2:740m | Local | 21.0 ±8.0 | 100 | — | 5.8 s | — |
| qwen3-embedding:0.6b | Local | 20.4 ±7.9 | 100 | — | 8.4 s | — |
| paraphrase-multilingual | Local | 19.8 ±7.8 | 100 | — | 2.6 s | — |
| snowflake-arctic-embed2 | Local | 19.4 ±7.8 | 100 | — | 5 s | — |
| granite-embedding:278m | Local | 18.4 ±7.6 | 100 | — | 2.9 s | — |
| gemini-embedding-001 | API | 18.4 ±7.6 | 100 | — | 4.5 s | $0.062 |
Amber edge and bar: tied with the best (score ≥ best − its 95% margin, harness rule). Only rows with the task's full n are compared.