Data tables
Benchmark runs
One dataset, typed columns, sorting on every header and search past ten rows.
All runs by p95 latency
24 rows
| Model | Provider | Tokens | Saving | Cost / 1M | p95 | Status | Coverage | Released | Six-week trend |
|---|---|---|---|---|---|---|---|---|---|
| atlas-large | Northwind | 1.8k | 35% | $0.12 | 180 ms | pass | 48% | Jan 1, 2026 | 30, 47, 64, 81, 38, 55 |
| helix-deep | Helix | 120.6k | 50% | $4.07 | 235 ms | pass | 67% | Apr 26, 2026 | 45, 62, 79, 36, 53, 70 |
| atlas-mini | Northwind | 9.7k | 72% | $0.65 | 277 ms | pass | 77% | Jun 12, 2026 | 43, 60, 77, 34, 51, 68 |
| iris-vision | Iris | 128.5k | 87% | $0.60 | 332 ms | pass | 96% | Sep 9, 2026 | 58, 75, 32, 49, 66, 83 |
| beacon-7b | Harbor Labs | 17.6k | 49% | $1.18 | 374 ms | fail | 54% | Feb 23, 2026 | 56, 73, 30, 47, 64, 81 |
| iris-text | Iris | 136.4k | 64% | $1.13 | 429 ms | flaky | 73% | May 20, 2026 | 71, 88, 45, 62, 79, 36 |
| beacon-13b | Harbor Labs | 25.6k | 86% | $1.71 | 471 ms | pass | 83% | Jul 6, 2026 | 69, 86, 43, 60, 77, 34 |
| juniper-1 | Juniper | 144.3k | 41% | $1.66 | 526 ms | pass | 50% | Jan 3, 2026 | 84, 41, 58, 75, 32, 49 |
| cedar-pro | Cedar AI | 33.5k | 63% | $2.24 | 568 ms | flaky | 60% | Mar 17, 2026 | 82, 39, 56, 73, 30, 47 |
| juniper-2 | Juniper | 152.3k | 78% | $2.19 | 623 ms | pass | 79% | Jun 14, 2026 | 37, 54, 71, 88, 45, 62 |
| cedar-lite | Cedar AI | 41.4k | 40% | $2.77 | 665 ms | pass | 89% | Aug 28, 2026 | 35, 52, 69, 86, 43, 60 |
| kestrel-edge | Kestrel | 160.2k | 55% | $2.72 | 720 ms | pass | 56% | Feb 25, 2026 | 50, 67, 84, 41, 58, 75 |
| delta-coder | Delta | 49.3k | 77% | $3.30 | 762 ms | pass | 66% | Apr 11, 2026 | 48, 65, 82, 39, 56, 73 |
| kestrel-cloud | Kestrel | 168.1k | 92% | $3.25 | 817 ms | fail | 85% | Jul 8, 2026 | 63, 80, 37, 54, 71, 88 |
| delta-chat | Delta | 57.2k | 54% | $3.83 | 859 ms | fail | 95% | Sep 22, 2026 | 61, 78, 35, 52, 69, 86 |
| lumen-r | Lumen | 176k | 69% | $3.78 | 914 ms | pass | 62% | Mar 19, 2026 | 76, 33, 50, 67, 84, 41 |
| ember-xl | Ember | 65.2k | 91% | $0.36 | 956 ms | pass | 72% | May 5, 2026 | 74, 31, 48, 65, 82, 39 |
| lumen-q | Lumen | 183.9k | 46% | $0.31 | 1 s | pass | 91% | Aug 2, 2026 | 89, 46, 63, 80, 37, 54 |
| ember-s | Ember | 73.1k | 68% | $0.89 | 1.1 s | pass | 49% | Jan 16, 2026 | 87, 44, 61, 78, 35, 52 |
| fjord-base | Fjord | 81k | 45% | $1.42 | 1.1 s | flaky | 78% | Jun 27, 2026 | 40, 57, 74, 31, 48, 65 |
| fjord-instruct | Fjord | 88.9k | 82% | $1.95 | 1.2 s | pass | 55% | Feb 10, 2026 | 53, 70, 87, 44, 61, 78 |
| granite-v2 | Quarry | 96.8k | 59% | $2.48 | 1.3 s | pass | 84% | Jul 21, 2026 | 66, 83, 40, 57, 74, 31 |
| granite-v3 | Quarry | 104.7k | 36% | $3.01 | 1.4 s | pass | 61% | Mar 4, 2026 | 79, 36, 53, 70, 87, 44 |
| helix-fast | Helix | 112.7k | 73% | $3.54 | 1.5 s | fail | 90% | Aug 15, 2026 | 32, 49, 66, 83, 40, 57 |
No rows match the search.
Top five savings
| Model | Saving | Status |
|---|---|---|
| kestrel-cloud | 92% | fail |
| ember-xl | 91% | pass |
| iris-vision | 87% | pass |
| beacon-13b | 86% | pass |
| fjord-instruct | 82% | pass |
Release checklist
| step | owner | hours |
|---|---|---|
| Freeze the dataset | data | 3 |
| Re-run the suite | eval | 6.5 |
| Publish the report | docs | 1.25 |