Models × HardwareWhat exists · Stop 7 of 9

What runs where, and how fast.

Pick an open-weights model and see which chips can hold it, how many you need, how fast each user’s answer streams, how many users it serves at once and what the hardware costs, from a desk-side DGX Spark to a rack of Blackwells. Every number is a first-order estimate built from the model’s real architecture.

After this page you can answer
  1. Will a given open model fit on a given chip, and how many chips does it need?
  2. How fast will it stream, and for how many users at once?
  3. What does that speed cost in hardware?
Data last revised November 2026
The calculator

Fit it, then feed it.

Two questions decide everything. Does it fit? That’s the weights plus every user’s KV cache against memory capacity. How fast does it go? For decode, that’s mostly bytes moved per token against memory bandwidth. Quantization shrinks both; long contexts and many users grow the cache.

Start with a question

Or build your own Every control below updates the link in your address bar, so a setup can be shared.

Parameters
117B · 5.1B active per token
Released as
MXFP4 experts · 65 GB checkpoint
Context window
131k tokens
Attention
18 full + 18 sliding-window (128) layers
KV cache
21 MB per 1k tokens at FP8
Quality index
45.8 benchleader ↗ weights ↗
Weights As released65 GB
The official checkpoint, in whatever mix of formats the lab shipped.
KV cache FP8
Fastest for each user419 tok/s1 × MI455X (Helios)
Cheapest hardware that runs it$3k1 × Ryzen AI Max+ 395 (128 GB) · 44 tok/s each
Memory to hold it66 GB65 GB weights + 606 MB KV for 1 user
datacenter desk-side best speed for the moneypriced hardware that fits · both axes log
50100200500$5k$10k$20k$50k$100kHARDWARE COST · purchase price of the whole setup →TOK / S PER USERMI455X (Helios)MI355XMI300X3× Arc Pro B70Ryzen AI Max+ 395 (128 GB)
HardwareSetupMemory usedTok/s per userTotal tok/sMax usersHardware$ / M tokens
MI455X (Helios)432 GB · 23.3 TB/s1 ×66 GB of 389 GB419memory-bound419533$73k$1.84
Vera Rubin NVL72288 GB · 19.2 TB/s1 ×66 GB of 259 GB410memory-bound410319$97k$2.50
Jalapeño232 GB · 15.4 TB/s1 ×66 GB of 209 GB399memory-bound399236not sold—
MTIA 400288 GB · 9.2 TB/s1 ×66 GB of 259 GB365memory-bound365319not sold—
TPU 8i288 GB · 8.6 TB/s1 ×66 GB of 259 GB360memory-bound360319not priced yet—
B200180 GB · 8 TB/s1 ×66 GB of 162 GB354memory-bound354159$40k$1.31
B300270 GB · 8 TB/s1 ×66 GB of 243 GB354memory-bound354293$53k$1.71
GB200 NVL72186 GB · 8 TB/s1 ×66 GB of 167 GB354memory-bound354168$48k$1.57
GB300 NVL72278 GB · 8 TB/s1 ×66 GB of 250 GB354memory-bound354305$53k$1.75
MI355X288 GB · 8 TB/s1 ×66 GB of 259 GB354memory-bound354319$28k$0.99
TPU7x Ironwood192 GB · 7.37 TB/s1 ×66 GB of 173 GB347memory-bound347177$12.00/hrent only$9.60rented
DGX Station GB300252 GB · 7.1 TB/s1 ×66 GB of 227 GB344memory-bound344266$95k$3.11
Maia 200216 GB · 7 TB/s1 ×66 GB of 194 GB343memory-bound343213not sold—
TPU 8t216 GB · 6.53 TB/s1 ×66 GB of 194 GB336memory-bound336213not priced yet—
MI325X256 GB · 6 TB/s1 ×66 GB of 230 GB328memory-bound328272sold, price not public—
MI300X192 GB · 5.3 TB/s1 ×66 GB of 173 GB316memory-bound316177$13k$0.52
Trainium3144 GB · 4.9 TB/s1 ×66 GB of 130 GB308memory-bound308106cloud only—
H200141 GB · 4.8 TB/s1 ×66 GB of 127 GB306memory-bound306101$27k$1.03
Gaudi 3128 GB · 3.7 TB/s1 ×66 GB of 115 GB278memory-bound27882$16k$0.73
H10080 GB · 3.35 TB/s1 ×66 GB of 72 GB267memory-bound26711$33k$1.40
Trainium296 GB · 2.9 TB/s1 ×66 GB of 86 GB251memory-bound25134$2.23/hrent only$2.48rented
TPU v5p95 GB · 2.77 TB/s1 ×66 GB of 86 GB245memory-bound24533$4.20/hrent only$4.76rented
TPU v5e16 GB · 820 GB/s8 ×66 GB of 115 GB227memory-bound22782$9.60/hrent only$12rented
TPU v6e Trillium32 GB · 1.64 TB/s4 ×66 GB of 115 GB227memory-bound22782$10.80/hrent only$13rented
GeForce RTX 509032 GB · 1.79 TB/s3 ×66 GB of 86 GB218memory-bound21834$19k$1.27
GeForce RTX 4090 (used)24 GB · 1.01 TB/s4 ×66 GB of 86 GB203memory-bound20334$13k$1.03
GeForce RTX 3090 (used)24 GB · 936 GB/s4 ×66 GB of 86 GB199memory-bound19934$6k$0.59
RTX PRO 6000 Blackwell96 GB · 1.79 TB/s1 ×66 GB of 86 GB195memory-bound19534$17k$1.05
Inferentia232 GB · 820 GB/s4 ×66 GB of 115 GB192memory-bound19282$4.32/hrent only$6.26rented
Radeon AI PRO R970032 GB · 640 GB/s3 ×66 GB of 86 GB157memory-bound15734$7k$0.68
Arc Pro B7032 GB · 608 GB/s3 ×66 GB of 86 GB154memory-bound15434$6k$0.57
Mac Studio M5 Ultra (512 GB)512 GB · 1.2 TB/s1 ×66 GB of 384 GB152memory-bound152525$22k$1.68
Mac Studio M3 Ultra (512 GB)512 GB · 819 GB/s1 ×66 GB of 384 GB116memory-bound116525$9k$1.04
MacBook Pro M5 Max (128 GB)128 GB · 614 GB/s1 ×66 GB of 96 GB93memory-bound9350$7k$0.82
DGX Spark128 GB · 273 GB/s1 ×66 GB of 109 GB46memory-bound4671$7k$1.71
Ryzen AI Max+ PRO 495 (192 GB)192 GB · 273 GB/s1 ×66 GB of 159 GB46memory-bound46155$7k$1.66
Ryzen AI Max+ 395 (128 GB)128 GB · 256 GB/s1 ×66 GB of 109 GB44memory-bound4471$3k$0.95
Doesn’t fit
Mac mini M5 Pro (64 GB)Needs 66 GB; one device holds 48 GB.

Estimates, not benchmarks: decode time per token = max(bytes read ÷ 65% of peak bandwidth, FLOPs ÷ 50% of peak compute) plus a fixed per-layer cost, with bandwidth pooled only across a fast scale-up link. Real engines usually land below these numbers, most of all at batch 1 on many GPUs. $ / M tokens = hardware amortized over 3 years running 24/7 plus power at $0.15/kWh, or the cheapest public hourly rental for chips you can only rent, at your number of users.

Figure 7.1Every chip on the site ranked for the model you pick: how many it takes to hold the weights and every user’s cache, how fast each answer streams, and what a million tokens costs. Change the precision, the context or the number of users and watch the ranking reorder.
Under the hood

How the estimate works.

Memory: will it fit?

Weights are parameters times bytes per parameter. The KV cache depends on the attention design, which we read from each model’s config.json.

need = params × bits ÷ 8
+ users × KV per token × context

Latent attention (MLA) caches one small vector per layer; sliding-window layers stop growing at their window; linear-attention and Mamba layers keep a fixed state. That’s why two 30B models can differ 9× in cache size. Datacenter GPUs group in powers of two up to one 8-GPU server, then by whole servers; desk-side machines go one card at a time.

Speed: how fast per user?

Each decode step streams the active weights and every user’s KV cache through the chip once, and does 2 FLOPs per active parameter per user.

step = max(bytes ÷ bandwidth,
FLOPs ÷ compute) + per-layer cost

Mixture-of-experts models read only the active experts for one user, but a batch touches more experts, so we count the share of weights a batch of b tokens hits: 1 − (1 − active/total)b. Bandwidth adds up only across a fast link like NVLink; beyond it, servers pipeline for throughput, not speed.

Cost: what does a token cost?

Hardware prices are street or reported prices per chip or card. Rack-scale GPUs are the rack’s price ÷ 72, which includes its CPUs, switches and cooling; desk-side machines are the whole system.

$/hour = price ÷ (3 years × 8,760 h)
+ watts × $0.15 per kWh

Divide by tokens per hour at your number of users and you get $ per million tokens. One user wastes most of a big GPU; raise the users slider and watch the cost collapse until the KV cache runs out of room. Cloud rental prices are on the Silicon page.

What the estimate leaves out: prefill time (the first token), attention FLOPs, speculative decoding (which can add 1.5–3× per user), engine overheads that vary by framework, and quality loss from quantization. See the serving playbook for what each technique changes.

Architecture × hardware

The architecture writes the bill.

Two models with similar scores can need very different hardware. Four choices in a model’s config decide it: how many parameters it stores, how many each token uses, how its attention remembers the conversation, and the precision of every number. Each one lands on a different line of the spec sheet.

Chip
Conversation length
  1. Seed-OSS-36B-InstructDense · 36B · full attention (GQA)
    45 tok/s1 × H100 to hold 40.3 GB
  2. Gemma 4 31BDense · 31B · mostly sliding window
    55 tok/s1 × H100 to hold 32.1 GB
  3. Qwen3.8 27BDense · 27B · mostly linear attention
    60 tok/s1 × H100 to hold 28.1 GB
  4. gpt-oss-120bMoE · 5.1B of 117B active · half sliding window
    204 tok/s2 × H100 to hold 118 GB
  5. Qwen3.5-122B-A10BMoE · 10B of 122B active · mostly linear attention
    139 tok/s2 × H100 to hold 122 GB
  6. DeepSeek V3.2MoE · 37B of 671B active · latent attention (MLA)
    120 tok/s16 × H100 to hold 672 GB
  7. DeepSeek V4 FlashMoE · 13B of 284B active · compressed attention
    172 tok/s4 × H100 to hold 284 GB

active weights that user’s KV cache · speeds are first-order estimates for one user, as in the calculator above

Show the arithmetic for gpt-oss-120b
weights read = 5.1B active params × 1 byte = 5.1 GB

A mixture-of-experts token only passes through the experts its router picks, so it reads 5.1B of 117B.

KV read = 18,432 values × 32K tokens + 18,432 × 128 (window) = 606 MB

Half its layers only look back 128 tokens, so their cache stops growing.

tokens/s ≈ 2 × 3.35 TB/s × 65% ÷ 5.7 GB ≈ 763, about 204 after the fixed cost of each of its 36 layers

2 chips split the model and read their shares at the same time, so their bandwidth adds up.

Figure 7.2Seven real models at the same precision (FP8 weights and cache), on the chip and conversation length you pick. Each bar is what one user’s token reads from memory, to scale: active weights in cyan, that user’s KV cache in magenta. Short bars stream fast; the right-hand column says what it takes to hold the whole model.
Stored parameters → memory capacity

Everything has to be loaded.

Every parameter sits in memory whether a token uses it or not. DeepSeek V3.2 activates 37B parameters per token but stores 671B: about 690 GB at FP8, more than a whole 8-GPU H100 server holds. Capacity, not speed, is often what forces a second server.

Active parameters → bandwidth and compute

Each token reads only what it uses.

Decoding reads the active weights once per step, and every token costs 2 FLOPs per active parameter. gpt-oss-120b reads 5.1B per token, so a DGX Spark’s 128 GB of slower memory runs it about as fast as a dense 5B model. The catch: a large batch wakes up most of the experts, so busy servers end up reading nearly the whole model every step.

Attention design → KV cache

Memory of the conversation.

Full attention keeps a key and a value for every head in every layer; grouped-query attention shares them across heads, often 8× smaller; latent attention keeps one compressed vector per layer; sliding-window layers stop growing at their window; linear and Mamba layers keep a fixed state. At 128K tokens Seed-OSS-36B keeps about 17 GB per user at FP8, DeepSeek V4 Flash under half a gigabyte. That decides how many conversations share a GPU, and how long ones slow down.

Precision → everything, if the chip speaks it

Fewer bits, fewer bytes.

FP8 halves every byte against BF16 and 4-bit halves it again, shrinking both what must fit and what each token reads. Fast maths in those formats needs hardware support: FP8 from Ada, Hopper and MI300, FP4 from Blackwell and MI355X. On older chips 4-bit weights still save memory and bandwidth, but are expanded to 16 bits for the maths, so prefill gets no faster.

So the right chip depends on the model: big mixtures-of-experts want capacity (MI355X and MI455X at 288 and 432 GB, or a 72-GPU rack); long conversations want capacity for the cache and bandwidth to read it; desk-side machines with lots of slower unified memory suit models with few active parameters.

The models

39 open-weights models, by size.

The best-scoring open-weights releases on benchleader.com, with architecture read from each model’s Hugging Face config. Select a name to load it into the calculator. Model not on the list? Paste any Hugging Face model id into Will it fit?, our Hugging Face Space. It reads that model’s config straight from the Hub and ranks 13 chips with the same estimate.

ModelParamsActiveReleased asContextKV · 1k tok (FP8)AttentionIndex
Moonshot AI · 2026-072.8T104BMXFP4 experts 1.6 TB1M14 MB24 MLA layers + 69 linear (KDA) layers61.5
Moonshot AI · 2026-102.8T104BMXFP4 experts 1.6 TB1M17 MB24 sparse MLA (DSA) layers + 69 linear (KDA) layers61.3
Alibaba · 2026-082.4T95BBF16 4.9 TB262k47 MB23 attention + 69 linear (Gated DeltaNet) layers51.4
DeepSeek · 2026-081.6T49BFP4 experts + FP8 865 GB1M9 MBcompressed attention (30 layers at 1/4, 31 at 1/128) + 128-token window (estimated)56.0
Xiaomi · 2026-091.0T42BFP4 experts + FP8 566 GB1M45 MB10 full + 60 sliding-window (128) layers56.8
Moonshot AI · 2026-041T32BINT4 experts 595 GB262k35 MBMLA on all 61 layers59.3
Mistral AI · 2026-101T52BFP8 1.1 TB1M42 MBMLA + indexer; 63 of 72 layers keep their own KV52.0
Thinking Machines · 2026-07975B41BBF16 1.9 TB66k138 MB11 global + 55 local (512-token) layers50.0
Zhipu AI · 2026-08744B40BFP8 756 GB1M48 MBMLA on all 78 layers; sparse-attention indexer on 2158.8
Zhipu AI · 2026-02744B40BBF16 1.5 TB203k55 MBMLA + sparse-attention indexer on all 78 layers60.2
DeepSeek · 2025-12671B37BFP8 690 GB164k43 MBMLA + sparse-attention indexer on all 61 layers56.3
DeepSeek · 2026-10552B16BFP4 experts + FP8 510 GB1M4 MB4 shared compressed KV caches (3 at 1/2, 1 full) + 128-token window (estimated)58.0
NVIDIA · 2026-06550B55BBF16 1.1 TB1M6 MB12 attention layers among 108 Mamba-2 / MoE layers55.4
Reflection · 2026-10501B23BBF16 1.0 TB1M64 MB9 full + 43 sliding-window (512) layers54.2
MiniMax · 2026-06428B23BBF16 854 GB1M61 MBGQA on all 60 layers with block-sparse attention (estimated)56.1
MiniMax · 2026-10428B23BNVFP4 experts + MXFP8 250 GB1M61 MBGQA on all 60 layers with block-sparse attention (estimated)53.5
Alibaba · 2026-02397B17BBF16 807 GB262k15 MB15 attention + 45 linear (Gated DeltaNet) layers56.4
Zhipu AI · 2026-08320B18BFP8 328 GB1M7 MB11 sparse MLA layers + 34 linear (KDA) layers56.5
Xiaomi · 2026-09309B15BFP4 experts + FP8 173 GB1M24 MB9 full + 39 sliding-window (128) layers54.8
Tencent · 2026-07295B21BBF16 598 GB262k164 MBGQA on all 80 layers (8 KV heads)55.7
DeepSeek · 2026-07284B13BFP4 experts + FP8 160 GB1M6 MBcompressed attention (21 layers at 1/4, 20 at 1/128) + 128-token window (estimated)57.1
StepFun · 2026-05196B11BBF16 403 GB262k59 MB12 full + 33 sliding-window (512) layers53.0
Alibaba · 2026-08125B6BBF16 360 GB262k12 MB12 attention + 36 linear (Gated DeltaNet) layers (estimated)53.9
Ant Group · 2026-08124B5.1BBF16 255 GB262k4 MBabout 7 MLA layers + 35 linear (KDA) layers (estimated)51.7
Alibaba · 2026-02122B10BBF16 250 GB262k12 MB12 attention + 36 linear (Gated DeltaNet) layers54.1
NVIDIA · 2026-03120B12BFP8 128 GB1M4 MB8 attention layers among 88 Mamba-2 / MoE layers51.1
OpenAI · 2025-08117B5.1BMXFP4 experts 65 GB131k21 MB18 full + 18 sliding-window (128) layers45.8
Zhipu AI · 2025-07106B12BBF16 221 GB131k94 MBGQA on all 46 layers (8 KV heads)46.7
ByteDance · 2025-0836BdenseBF16 72 GB524k131 MBGQA on all 64 layers (8 KV heads)49.1
Alibaba · 2026-0435B3BBF16 72 GB262k10 MB10 attention + 30 linear (Gated DeltaNet) layers51.8
Google · 2026-0431BdenseBF16 63 GB262k430 MB10 global + 50 sliding-window (1024) layers49.8
Zhipu AI · 2026-0130B3BBF16 62 GB203k27 MBMLA on all 47 layers51.0
NVIDIA · 2025-1230B3.5BBF16 63 GB1M3 MB6 attention layers among 52 Mamba-2 / MoE layers47.4
Meta · 2026-0829.6BdenseBF16 60 GB131k27 MB13 full + 39 sliding-window (2048) layers54.4
Alibaba · 2026-0827BdenseBF16 56 GB262k33 MB16 attention + 48 linear (Gated DeltaNet) layers53.1
Google · 2026-0425.2B3.8BBF16 52 GB262k108 MB5 global + 25 sliding-window (1024) layers56.1
OpenAI · 2025-0821B3.6BMXFP4 experts 14 GB131k14 MB12 full + 12 sliding-window (128) layers45.6
Alibaba · 2026-029BdenseBF16 19 GB262k16 MB8 attention + 24 linear (Gated DeltaNet) layers50.4
Alibaba · 2026-024BdenseBF16 9.3 GB262k16 MB8 attention + 24 linear (Gated DeltaNet) layers49.3
Sources & notes

Model list, release dates and quality index from benchleader.com (refreshed 14 November 2026). Parameter counts are from each model card; checkpoint sizes, layer layouts and context windows from the Hugging Face repositories, checked 30 October and 14 November 2026. Throughput and cost figures are first-order estimates, not measurements: real engines differ by framework, batch shape and kernel support. KV-cache sizes marked estimated are our reading of a new architecture’s config (DeepSeek V4 and V4.1 compressed attention, MiniMax M3 sparse attention, Ling 3.0, Qwen3.8 Flash Next).

Recap

The short version

  1. Weights take parameters × bits ÷ 8 bytes. Add a KV cache per user and compare with memory capacity.
  2. Per-user speed is roughly bandwidth ÷ the bytes read per token; batching multiplies the total.
  3. The cheapest hardware that fits is rarely the fastest. The chart shows the trade.
Check yourself

A model has 120 billion parameters. Stored at 4 bits each, about how much memory do the weights alone need?

Pick one. The reasoning shows either way.