NEWGPU owners: host free demos on your card and earn a bigger share of every paid job. See how →

For developers

Live API benchmarks

How fast and how reliably each open-weight model answered on the legit2compute API, measured on real traffic, next to its price.

How to read it. Each row is one model over its last 24 hours of requests, or its last 7 days when the day was quiet. Nothing here is a lab test, and nothing is published without at least 20 requests behind it. Updated 2026-11-21 20:40 UTC.

By model

Price is dollars per million tokens, input / output. Response time is the whole answer.

ModelPrice in / outMedian response95th percentileOutput tokens / sAnsweredRequests
Qwen3.8 27B (vision + text)
qwen3.8:27b
$0.15 / $1.80 2.09 s9.33 s 61.898.1% 1,476 · 24 h
Ornith 1.5 35B (vision, 256K)
ornith-1.5:35b
$0.12 / $0.80 2.05 s13.8 s 57.196.3% 999 · 24 h
Llama 3.1 8B
llama3.1
$0.030 / $0.050 74 ms480 ms —100% 51 · 24 h
MiniCPM-V 2.6 8B (vision)
minicpm-v:latest
$0.040 / $0.060 1.80 s4.04 s 43.8100% 65 · 24 h
Ornith 1.5 9B (vision, 256K)
ornith-1.5:9b
$0.080 / $0.14 4.85 s28.7 s 2892.6% 81 · 7 d
Whisper large-v3 turbo
whisper-large-v3-turbo
$0.030 / audio hour 345 ms120.4 s —100% 49 · 24 h
Bonsai 2 9B (vision)
bonsai-2:9b
$0.064 / $0.10 —— —— 3 · 7 d
MiMo-V2.6 9B (vision)
mimo-v2.6:9b
$0.030 / $0.10 —— —— 3 · 7 d
LFM3 1.7B (text)
lfm3:1.7b
$0.040 / $0.060 —— —— 2 · 7 d
Granite 4.2 8B (text)
granite4.2:8b
$0.050 / $0.16 —— —— 1 · 7 d

11 more models in the catalog had no requests this week.

Standard and Private, side by side

Last 7 days. Private keeps an account's requests on machines we own.

ModelTokens / s, standardTokens / s, PrivateMedian, standardMedian, PrivateAnswered, standardAnswered, Private
qwen3.8:27b 54.161.2 4.22 s2.46 s 89.2%100%
ornith-1.5:35b 32.846.1 6.70 s1.93 s 67.8%100%
ornith-1.5:9b 28.826.9 7.62 s3.50 s 95.3%89.5%

Read it from code

curl https://www.legit2compute.com/benchmarks.json

No key. Each model carries last_24h and last_7d, each with the overall figures plus standard and private; the method block says how every field is measured.

Questions

Where do the numbers come from?

From the API gateway's own request log: every request customers and pipelines sent in the window, not a synthetic test. Our own test calls are left out, and a request the caller abandoned is not counted.

Why is there no time to first token?

The gateway does not record it yet, so it is not published. Response time here is the whole answer, which depends on how long the answer was; tokens per second is the fairer way to compare models.

What does "answered" mean?

The share of requests that got an answer. A model that was not loaded when asked answers 503 with "warming": true while it starts, and that counts against it here. Models marked always on in the catalog stay loaded.

What is the difference between standard and Private?

Private is a switch on the developer dashboard that keeps an account's requests on machines we own. Standard is every other request. The two carry different workloads, so compare tokens per second rather than response time.

Can I use this data?

Yes. benchmarks.json is public, needs no key, allows cross-origin reads and refreshes every 5 minutes. Please link to this page as the source.

Run your own numbers.

Keep the OpenAI SDK you use, change the base URL, and time it yourself.