Ollama CPU-only performance: tokens/s, prompt speed, load time and RAM on 5 servers

Ollama CPU-only performance on a rented VPS is usable for small models: Llama 3.2 3B generated 13.5 tokens/s on a 4 vCPU Hetzner CX33 and 21.2 tokens/s on the 8 vCPU CX43, with no GPU. Llama 3.1 8B ran at 6.3 and 9.7 tokens/s on the same two servers. The slowest server, a 2 vCPU Intel CX23, managed 5.8 tokens/s on 3B.

Why this matters now: most published CPU numbers come from desktop Ryzen or mini PCs, not the shared-vCPU cloud servers people actually rent for Ollama. This report gives the full data from five such servers, measured on 5 November 2026, plus the exact commands so you can rerun it. It belongs to our local AI section.

Tested in the Lab Hetzner CX23, CX33, CX43, CAX11, CAX21 · Nov 2026

The results

Three Llama models, five servers, Ollama 0.40.14 on Ubuntu 24.04. Units, explained once:

• Generation tokens/s: how fast the answer is written. A token is a word or a piece of a word.
• Prompt tokens/s: how fast Ollama reads your input before it starts writing.
• Load time: how long the model takes to go from disk into memory on the first request.
• RAM used: system memory in use with the model loaded, from free -m, in MiB (1 MiB = 1,048,576 bytes). It includes the operating system and an idle Docker daemon our script installed earlier.
• vCPU: a virtual CPU core. On these shared plans the physical cores are shared with other customers.

Measured by AI Hosting Lab on 2026-11-05 · 5 servers · method · raw data

Generation speed, Llama 3.2 3B

Generation speed, Llama 3.2 1B

Prompt speed vs generation speed

On a CPU these are two different jobs. Reading the prompt processes many tokens in one batch, so it keeps all cores busy. Writing the answer produces one token at a time, and each token needs the whole model read from RAM again. That is why the numbers are so far apart: on the CX43, Llama 3.2 3B read at 1161.3 tokens/s and wrote at 21.2 tokens/s.

What it means in practice: a short question gets a fast first word, and generation speed decides how long the full answer takes. A 256-token answer at 13.5 tokens/s (CX33, 3B) takes about 19 seconds; at 5.8 tokens/s (CX23) about 44 seconds. If your workload is mostly reading (classifying emails, extracting fields from documents, short yes/no answers), prompt speed matters more than the generation number.

One caveat we want to be clear about: our prompt is short (a three-sentence question). Long prompts of thousands of tokens can behave differently. We have not measured long-context prompt speed yet.

What CPU type changes

We ran sysbench on the same servers. Its single-thread CPU score (events per second, higher is better) puts the old Intel Xeon in the CX23 far behind everything else:

The CX23 scored 913.87 events/s, the ARM CAX11 3248.26, about 3.55 times as much. If CPU speed decided Ollama speed, the CAX11 would run away with it. It did not:

The pattern fits the explanation above. Generation on the bigger 3B model moved almost in step with the memory score, which was nearly the same on both servers, not with the CPU score. The smaller 1B model and prompt reading, where compute matters more, showed bigger gaps. Our memory test is a single-thread sysbench run, not a full bandwidth measurement, so treat this as a strong hint, not proof.

The practical lesson: a CPU benchmark score is a poor shopping guide for Ollama. More vCPUs and a newer platform helped (below); a faster single core on its own helped little on 3B. ARM against x86 in detail is in Ollama on ARM servers.

What more threads change

We did not set any thread option, so Ollama used its default. Doubling vCPUs inside the same CPU family gave these gains:

From 2 to 4 ARM vCPUs, 3B generation went up 1.88 times. From 4 to 8 AMD vCPUs it went up 1.57 times. More cores keep helping on these sizes, with diminishing returns. Note the CX43 also had a higher single-thread score than the CX33 (3470.38 vs 1620.94 events/s) although both report the same AMD EPYC-Rome CPU, so part of its gain may come from the host, not only from the extra cores. One run per server cannot separate the two.

Load time and RAM

Load times on the first request: 1.7 to 4.8 s for 1B, 3.7 to 8.8 s for 3B, 7.9 to 12.3 s for 8B. The CX23 was slowest on 3B (8.8 s) and the CAX21 on 8B (12.3 s). Ollama keeps a model loaded for 5 minutes after the last request by default, according to its FAQ, so you pay this once per idle gap, not per request.

RAM is where CPU-only Ollama gets decided. System RAM used with the model loaded was 2001 to 2315 MiB for 1B, 3002 to 3282 MiB for 3B and 5796 to 6023 MiB for 8B. The Ollama library lists the default 3B file at 2.0 GB and the 8B file at 4.9 GB, both Q4_K_M; the rest is the operating system, the runtime and the context buffer. Per-model detail is in Llama 3.2 3B requirements and Llama 3.1 8B requirements.

Method: exactly what we ran

Five fresh Hetzner Cloud servers in Helsinki (hel1), Ubuntu 24.04.4, kernel 6.8, no swap, all started within 30 seconds of each other on 5 November 2026. CPU steal over 30 s was 0.0 % on all five, so noisy neighbours did not distort this run. Ollama was installed natively (not Docker) with the official script. These lines are from our bench.sh:

curl -fsSL https://ollama.com/install.sh | sh
PROMPT="Explain in plain English what a virtual private server is and when a small business should use one instead of shared hosting. Give three concrete examples."
for m in llama3.2:1b llama3.2:3b llama3.1:8b; do
  ollama pull $m
  for run in 1 2; do
    curl -s --max-time 900 localhost:11434/api/generate -d "{\"model\":\"$m\",\"prompt\":\"$PROMPT\",\"stream\":false,\"options\":{\"num_predict\":256,\"temperature\":0,\"seed\":42}}"
  done
  curl -s localhost:11434/api/generate -d "{\"model\":\"$m\",\"keep_alive\":0}"
done

Run 1 gives the load time (load_duration) and the RAM reading, taken right after it. Run 2, with the model already in memory, gives the speeds: generation = eval_count / eval_duration, prompt = prompt_eval_count / prompt_eval_duration. The Ollama API documents all durations in nanoseconds. Temperature 0 and a fixed seed make both runs produce the same 256 tokens. The keep_alive: 0 call unloads each model before the next one loads.

Limits of this test: one run per server, one prompt, default 4096-token context, one request at a time, default quantization only, Llama models only. We have not tested Qwen or Mistral models, other quantizations or concurrent users yet. The general rules are on how we test.

Next step: turn these numbers into a purchase with our VPS for Ollama verdict by model size, including cost per million tokens on each server.