Local LLM Speed by Hardware: What llmspeed Gets Right

Last updated: October 18, 2026

Before you buy hardware for a local LLM, you want to know how fast it will feel. A Show HN project, llmspeed, now estimates that: pick one of 30 machines and one of 15 models and it streams text at the predicted tokens per second. The estimates are useful for ranking machines and misleading for agentic coding, because they describe a short prompt and coding agents live on long ones. This post covers how the numbers are built, a bandwidth check you can run yourself, and what to measure before you spend money.

We have not run any of these machines. Everything below is llmspeed's published method and table, the arithmetic is ours and labelled as such, and the numbers are estimates.

What llmspeed says it does#

The llmspeed page covers 30 GPUs, Macs and CPUs and 15 models, and labels each estimate by how much to trust it. Per its README:

• Measured: benchmarked on that exact hardware, model and quantization.
• Derived: interpolated from measured benchmarks of similar setups.
• Projected: calculated from specs, for example for announced hardware.

When no benchmark exists it falls back to first principles. Generation speed is bound by memory bandwidth, because every token reads the active weights once, and for mixture-of-experts models only the active parameters count. Prompt processing is bound by compute. Efficiency coefficients are calibrated against community benchmarks from r/LocalLLaMA, localscore.ai and llama.cpp discussions, with a slowdown as context grows. The README's own limitations section says drivers, engine versions, flash attention, power limits and thermals can move real numbers by 10 to 30 percent, and that multi-GPU setups are not simulated.

The project is young: the site's hardware pages and the README commit log show catalogs refreshed for October 2026 this week, and the author notes the repo was built with Claude. Treat it as a good first filter, not a spec sheet.

A bandwidth check you can redo#

The DGX Spark page lists 128 GB of memory and 273 GB/s of memory bandwidth, and gives per-model speeds. Two rows make the rule visible:

The ceiling is our back-of-envelope: bandwidth divided by the bytes read per token. A dense 27B model at roughly 4.85 bits per weight is about 16 GB; the 35B model with about 3B active parameters reads roughly 1.8 GB. Both estimates land near 42 percent of the ceiling, which is what a calibrated efficiency factor looks like. The takeaway does not depend on the exact percentage: on the same machine, the sparse model is about nine times faster than the dense one, and the memory-bandwidth number, not the memory size, decides it. That is the same point made for laptops in our RTX Spark bandwidth post, and the memory-size half is in How much VRAM do you need.

What a builder does Monday#

1. Use llmspeed to rank, not to promise. Race two machines on the model you actually want. If one is clearly ahead and the other cannot fit the model at all, you have learned something. If they are within 20 percent, the tool cannot separate them.
2. Look at the label. A "Projected" speed on unreleased hardware is a spec-sheet calculation. Wait for measurements.
3. Measure your own prefill. The README's contribution recipe is the same command you should run on any machine you are considering: llama-bench -m your-model.gguf -p 512 -n 128. The pp512 result is prompt processing and tg128 is generation. Agents care about the first number far more than chat does.
4. Check the failure case. If you plan to offload a model that does not fit, llmspeed marks it "Partial offload" and its DGX Spark table shows the cost: DeepSeek V4.1 Flash at Q2_K is estimated at 12 tok/s with a 1.2 s first token, against 0.3 s for the models that fit. For MoE offload tuning see llama.cpp MoE offload.
5. Pick the runtime second. Speeds above assume llama.cpp-style backends; your harness and runtime change the result, covered in Ollama vs LM Studio vs vLLM vs llama.cpp.

Where it breaks for coding agents#

llmspeed states that its estimates are for a short prompt on a single device. A coding agent sends a system prompt, tool definitions and file contents, often tens of thousands of tokens, on every turn. That makes time to first token (prefill) the wait you feel, and prefill is compute-bound, not bandwidth-bound. A machine that wins the tokens-per-second race can lose the "time until the agent starts typing" race. The tool does show a first-token time, but for the short-prompt case.

This is a gap in the tool as scoped, not a flaw: the author replied in the thread that benchmark sources rarely record the harness, so data is the limit. For picking a model that handles agent loops, see The Best Local Coding LLMs in 2026.

Who wins and loses#

Buyers win: a free, open-source (MIT) first filter beats a forum scroll. Hardware vendors with unified memory and high bandwidth look good on large sparse models; cards with fast bandwidth but small VRAM look worse the moment a model does not fit. The losers are marketing numbers quoted without a model, quantization and context length attached. Expect more of these tools to appear, and expect measured submissions, which the project accepts as pull requests, to matter more than the formulas.

What people are actually saying#

The Hacker News thread (256 points when we read it) was mostly constructive criticism:

• One commenter said their M2 Ultra 128 GB runs almost twice as fast as the tool reports for an M3 Ultra 512 GB, and wanted the estimates to be "somewhat correct." The author asked which model and quantization to investigate. We did not verify the claim.
• Another said prefill parallelism matters far more than output speed for agentic coding, the same gap described above.
• A commenter running company endpoints on H200 GPUs said different NVIDIA driver versions changed their results, especially with only 24 GB of VRAM, and linked a llama.cpp issue as an example. We did not open that issue.
• Others asked for multi-GPU, PCIe generation and NVLink options, harness and library versions per setup, and older cards like the RX 580.

The counter-case, from a commenter replying about harness overhead, is that a proper harness adds negligible startup cost, so the harness caveat may matter less than it sounds. Both sides are opinions in a comment thread, not measurements.

FAQ#

How fast is a local LLM on my hardware?#

Pick your machine and model on llmspeed, then confirm with llama-bench on a model you have downloaded. Treat the site's estimate as a range: its own README says real results can differ by 10 to 30 percent.

Why is a sparse model so much faster than a dense one on the same machine?#

Generation reads the active weights for every token. A sparse mixture-of-experts model with about 3B active parameters reads far fewer bytes than a dense 27B model, so on the same memory bandwidth it can be several times faster.

Does llmspeed work for coding agents?#

Only partly. Its estimates assume a short prompt on a single device, while agents send long prompts, so prefill speed and time to first token matter more than the tokens-per-second headline.

Continue Reading#

• How much VRAM do you need to run a local LLM in 2026? - the memory-size half of the buying decision
• RTX Spark for Local LLMs: Will It Run a Coding Agent? - the bandwidth arithmetic on an unreleased laptop
• The Best Local Coding LLMs in 2026 - what to run by hardware tier
• Ollama vs LM Studio vs vLLM vs llama.cpp - picking a runtime for agents
• llama.cpp MoE offload - when a model does not fit

Sources#

• llmspeed - read October 18, 2026
• llmspeed on DGX Spark 128GB - per-model estimates quoted above
• nicorobinreal/llmspeed README - method, labels, limitations
• Hacker News: Show HN: Llmspeed