For developers
How fast and how reliably each open-weight model answered on the legit2compute API, measured on real traffic, next to its price.
Price is dollars per million tokens, input / output. Response time is the whole answer.
| Model | Price in / out | Median response | 95th percentile | Output tokens / s | Answered | Requests |
|---|---|---|---|---|---|---|
Qwen3.8 27B (vision + text)qwen3.8:27b |
$0.15 / $1.80 | 2.09 s | 9.33 s | 61.8 | 98.1% | 1,476 · 24 h |
Ornith 1.5 35B (vision, 256K)ornith-1.5:35b |
$0.12 / $0.80 | 2.05 s | 13.8 s | 57.1 | 96.3% | 999 · 24 h |
Llama 3.1 8Bllama3.1 |
$0.030 / $0.050 | 74 ms | 480 ms | — | 100% | 51 · 24 h |
MiniCPM-V 2.6 8B (vision)minicpm-v:latest |
$0.040 / $0.060 | 1.80 s | 4.04 s | 43.8 | 100% | 65 · 24 h |
Ornith 1.5 9B (vision, 256K)ornith-1.5:9b |
$0.080 / $0.14 | 4.85 s | 28.7 s | 28 | 92.6% | 81 · 7 d |
Whisper large-v3 turbowhisper-large-v3-turbo |
$0.030 / audio hour | 345 ms | 120.4 s | — | 100% | 49 · 24 h |
Bonsai 2 9B (vision)bonsai-2:9b |
$0.064 / $0.10 | — | — | — | — | 3 · 7 d |
MiMo-V2.6 9B (vision)mimo-v2.6:9b |
$0.030 / $0.10 | — | — | — | — | 3 · 7 d |
LFM3 1.7B (text)lfm3:1.7b |
$0.040 / $0.060 | — | — | — | — | 2 · 7 d |
Granite 4.2 8B (text)granite4.2:8b |
$0.050 / $0.16 | — | — | — | — | 1 · 7 d |
11 more models in the catalog had no requests this week.
Last 7 days. Private keeps an account's requests on machines we own.
| Model | Tokens / s, standard | Tokens / s, Private | Median, standard | Median, Private | Answered, standard | Answered, Private |
|---|---|---|---|---|---|---|
qwen3.8:27b |
54.1 | 61.2 | 4.22 s | 2.46 s | 89.2% | 100% |
ornith-1.5:35b |
32.8 | 46.1 | 6.70 s | 1.93 s | 67.8% | 100% |
ornith-1.5:9b |
28.8 | 26.9 | 7.62 s | 3.50 s | 95.3% | 89.5% |
curl https://www.legit2compute.com/benchmarks.json
No key. Each model carries last_24h and last_7d, each with the overall figures plus standard and private; the method block says how every field is measured.
From the API gateway's own request log: every request customers and pipelines sent in the window, not a synthetic test. Our own test calls are left out, and a request the caller abandoned is not counted.
The gateway does not record it yet, so it is not published. Response time here is the whole answer, which depends on how long the answer was; tokens per second is the fairer way to compare models.
The share of requests that got an answer. A model that was not loaded when asked answers 503 with "warming": true while it starts, and that counts against it here. Models marked always on in the catalog stay loaded.
Private is a switch on the developer dashboard that keeps an account's requests on machines we own. Standard is every other request. The two carry different workloads, so compare tokens per second rather than response time.
Yes. benchmarks.json is public, needs no key, allows cross-origin reads and refreshes every 5 minutes. Please link to this page as the source.
Keep the OpenAI SDK you use, change the base URL, and time it yourself.