Cost per million tokens, self-hosted: measured on five CPU servers

The cost per million tokens on a self-hosted CPU server is the monthly price spread over everything the model writes. With our own measurements, the cheapest case was Llama 3.2 1B on a Hetzner CX33 running 24/7 at 0.17 € per million generated tokens. The most expensive full-time case was Llama 3.1 8B on the ARM CAX21 at 0.86 €. Use the server two hours a day instead and those figures rise twelvefold.

This matters now because Hetzner repriced its cloud plans in 2026, and the monthly price is half of this formula. Most pages on this topic assume a speed. We measured ours.

Tested in the Lab Hetzner CX23, CX33, CX43, CAX11, CAX21 · Nov 2026

This is part of our self-hosted LLM section. If you are still choosing hardware, read CPU vs GPU for LLM first.

Calculator

Pick a server and model we measured, set how many hours a day it actually generates, and enter an API price to compare. The calculator uses the formula below.

The API price field is in euros. Convert a dollar price yourself before you compare.

The formula

cost per 1M tokens = monthly price / (tokens/s × 3600 × hours per day × 30) × 1,000,000

24/7 case: monthly price / (tokens/s × 3600 × 24 × 30) × 1e6

Two inputs come from the Lab. Tokens/s is the generation speed we measured with Ollama, CPU only, for a 256-token answer at temperature 0. Monthly price is the Hetzner gross price (including VAT) from the Hetzner Cloud API, checked on 5 November 2026. A month is counted as 30 days.

Worked example: Llama 3.2 3B on the CX33 generated 13.5 tokens/s. 13.5 × 3,600 × 24 × 30 = 34,992,000 tokens a month. 10.27 € / 34,992,000 × 1,000,000 = 0.29 € per million.

Results for every server and model we measured

Our arithmetic, from the measured tokens/s and the monthly price of each server. Three utilisation cases: nonstop (24 h/day), a working day (8 h/day) and light use (2 h/day).

Missing rows are missing on purpose. Llama 3.1 8B did not run on either 4 GB server: the ARM CAX11 failed at once (out of memory) and the x86 CX23 was OOM-killed after about 50 s. The measured speeds behind this table:

Measured by AI Hosting Lab on 2026-11-05 · 5 servers · method · raw data

What the table tells you

The mid-size x86 server is the cost winner

For every model, the 4 vCPU CX33 had the lowest cost per token. The 8 vCPU CX43 is faster (21.2 vs 13.5 tokens/s with Llama 3.2 3B) but costs 19.35 € instead of 10.27 €, and its speed did not grow as fast as its price. Pick the CX43 for shorter waits, not for cheaper tokens.

ARM is not cheaper per token here

The ARM CAX21 costs 12.69 € and generated 12.2 tokens/s with the 3B model, so it came out at 0.40 € per million against 0.29 € on the x86 CX33. On the 2 vCPU tier it is closer: the CAX11 and the CX23 are almost level with the 3B model (0.43 € vs 0.44 €), and the CAX11 wins with the 1B model (0.19 € vs 0.25 €). See Hetzner CX vs CAX for the full comparison.

Utilisation decides everything

A VPS costs the same whether the model works or sits idle. Halve the hours and you double the cost per token. A server that answers a few questions a day can end up costing more per token than any API by a wide margin. GEICO's engineering team found the same effect on rented H100 GPUs: low utilisation multiplied cost per token by 2.5 to 24 times at realistic loads.

Comparison with API prices

Official pricing pages, opened on 5 November 2026. Prices are in US dollars per 1 million tokens. Our server figures are in euros; we did not convert them.

The fairest comparison is the same model. Llama 3.1 8B through DeepInfra cost $0.04 per million output tokens. Our cheapest self-hosted 8B figure was 0.63 € on the CX33 running nonstop. Even treating dollars and euros as equal, the API was more than 15 times cheaper per output token.

Against the small closed models, a fully busy CX33 running Llama 3.2 3B at 0.29 € per million is in the same range as their output prices. These are different models with different quality, so a lower price per token is not the same as a lower price per useful answer. Google also offers a free tier for Gemini 2.5 Flash-Lite.

The honest conclusion: on CPU, self-hosting does not save money per token against current API prices. People self-host for other reasons. Their data never leaves the server, the price is a fixed monthly figure, nothing is rate-limited, and the model cannot be changed or retired under them.

What the formula leaves out

• Prompt tokens. We count only generated tokens. Long inputs take time to read: 776.5 tokens/s for Llama 3.2 3B on the CX33 in our test. That time is billed by the month too.
• Parallel requests. We measured one request at a time. Ollama can handle several at once, but we have not measured whether total throughput rises on these CPUs, so we do not count on it.
• Other apps on the same server. If n8n or Supabase shares the box, part of the price belongs to them, and the model's share per token gets lower. It also has to fit in RAM: check with the VPS RAM calculator.
• Your time. Updates and monitoring are not in the per-token figure.
• GPUs. RunPod listed an RTX 4090 at $0.89 per hour on its pricing page today. We have not measured GPU throughput, so we cannot give a GPU cost per million yet. GPU tests are scheduled.

When the per-token math works in your favour

Self-hosted tokens get cheaper the more of the month the model spends writing. Three ways to get there:

• Queue the work. Collect jobs during the day (ticket tagging, summaries, classification) and let the model run through them overnight. On the CX33 with Llama 3.2 3B, our measured 13.5 tokens/s works out to about 1.17 million tokens per full day.
• Use the smallest model that does the job. On the same CX33, Llama 3.2 1B came out at 0.17 € per million at full use and Llama 3.1 8B at 0.63 €. Test the small one on your real task first.
• Share the server. If the box already runs n8n for your automations, the model uses capacity you pay for anyway. Just make sure both fit in RAM at the same time.

Skip self-hosting for cost reasons if your model only answers a handful of requests a day. At 2 hours of generation a day, even the best case in our table was 2.05 € per million, five times gpt-5-nano's listed $0.40 output price before currency conversion.

Current Hetzner prices used

From the Hetzner Cloud API, location hel1, checked 2026-11-05. "Incl. VAT" is what our Czech (EU consumer) account pays; businesses and buyers outside the EU usually pay the price excl. VAT. Primary IPv4 is billed separately.

Next step: see how these speeds feel in use, and when a GPU becomes worth renting, in CPU vs GPU for LLM.