Ollama on an ARM server: Hetzner CAX vs x86 CX on the same models

Ollama on an ARM server works with the same one-line install as on x86, and it is not slower by default. On our Hetzner servers the 2 vCPU ARM CAX11 generated Llama 3.2 3B at 6.5 tokens/s against 5.8 on the 2 vCPU Intel CX23. At 4 vCPU the result flipped: the AMD-based CX33 did 13.5 tokens/s, the ARM CAX21 12.2.

Why this matters now: ARM cloud servers are often sold as the cheap, efficient choice, and most ARM guides for local LLMs either show a Raspberry Pi or a 64-core cloud instance. We put two budget ARM servers next to three x86 servers of the same sizes and ran identical tests. This page belongs to the local AI section.

Tested in the Lab Hetzner CX23, CX33, CX43, CAX11, CAX21 · Nov 2026

Does Ollama support ARM64?

Yes. The official install.sh maps aarch64 to its arm64 build and refuses only unknown architectures. The Ollama Linux docs also list a separate ollama-linux-arm64.tar.zst package for manual installs. On both our ARM servers the script ran unchanged and reported the same version (0.40.14) as on x86. The models come from the same Ollama library tags; nothing ARM-specific to choose.

The ARM servers here are Hetzner's CAX line, built on Ampere Altra processors; lscpu reported the cores as Neoverse-N1. The x86 servers are the CX23 (Intel Xeon, Skylake generation) and the CX33 and CX43 (AMD EPYC-Rome). All ran Ubuntu 24.04 in Helsinki on 5 November 2026, CPU-only.

All results

Measured by AI Hosting Lab on 2026-11-05 · 5 servers · method · raw data

Head to head at the same size

Bold marks the winner of each pair. "Generation tokens/s" is how fast the answer is written; "prompt tokens/s" is how fast the input is read.

2 vCPU: ARM wins clearly

The CAX11 beat the CX23 on generation for both small models and read the 1B prompt at 835.5 tokens/s against 454. The reason is the CX23's older Intel CPU: its single-thread sysbench score was 913.87 events/s against 3248.26 for the CAX11. The one place the CX23 was faster is the 1B load time (3.5 s vs 4.8 s).

4 vCPU: x86 AMD wins on the models that matter

Against the AMD CX33 the picture changes. The CAX21 still had the higher single-thread score (3307.34 vs 1620.94 events/s) and won on the 1B model, but the CX33 was faster on 3B and 8B, read prompts faster on both, and loaded every model faster (8B: 10 s vs 12.3 s). The CX33 also had the higher single-thread memory score (4973.99 vs 4263.84 MiB/s). That fits what we see across all five servers: once the model is larger than a toy, generation speed follows memory more than CPU benchmarks. The detail is in our Ollama CPU-only performance report.

How ARM scales with more vCPUs

Going from the 2 vCPU CAX11 to the 4 vCPU CAX21 raised Llama 3.2 3B generation from 6.5 to 12.2 tokens/s (1.88 times) and prompt reading from 376.4 to 664.1 tokens/s. On the AMD side, doubling from the CX33 to the 8 vCPU CX43 gave 13.5 to 21.2 tokens/s. ARM scaled well from 2 to 4 vCPU, but it started from a lower point than AMD at 4 vCPU. We have not tested the larger CAX31 and CAX41 yet, so we cannot say whether 8 ARM vCPU would catch the CX43.

The 8B failure on 4 GB ARM, and what it means

Llama 3.1 8B downloaded fine on the CAX11 (19.6 s) and then failed on the first request with:

llama-server process has terminated: exit status 1: ggml_aligned_malloc: insufficient memory (attempted to allocate 4402.48 MB)

The server has 3805 MiB of RAM in total and no swap. The Ollama library lists the default 8B file at 4.9 GB, larger than this server's entire RAM. The ARM CAX21 with 8 GB ran the same model at 5.7 tokens/s using 5796 MiB. So the failure tells you about RAM, not about ARM: 4 GB servers are for 1B and 3B models.

The 4 GB x86 server failed too, which confirms it. On the Intel CX23, the Ollama fit check reported it would leave 1698 MiB free, then the kernel OOM killer stopped llama-server (anon RSS 3.5 GB), after about 50 seconds. So ARM failed fast and clearly, x86 failed late after looking like it would work. Same answer either way. With Llama 3.2 3B loaded, the CAX11 had 591 MiB of its RAM left, so even 3B leaves little room for other apps there.

What about price?

At Hetzner's current prices ARM is not the cheaper option for Ollama. From the Hetzner Cloud API on 5 November 2026: CAX11 7.25 € vs CX23 6.64 € a month, CAX21 12.69 € vs CX33 10.27 €. So:

• At 2 vCPU, the CAX11 costs 0.61 € more and is faster on everything that runs. Worth it.
• At 4 vCPU, the CAX21 costs 2.42 € more and is slower on 3B and 8B. Take the CX33, unless your stack is already ARM or you only run 1B models.

Hetzner's full x86 vs ARM comparison beyond Ollama is in Hetzner CX vs CAX. Other ARM clouds (AWS Graviton, Oracle Ampere A1 and others) are not yet tested in our Lab, so we make no claims about them.

Before you choose ARM

• Native Ollama needs nothing special on ARM. Other tools on the same server might: check that every Docker image you plan to run next to Ollama publishes an arm64 build.
• No GPU is involved on any of these servers. ARM vs x86 only matters for CPU inference.
• Our numbers are one run per server with Llama models at the default Q4_K_M quantization and a 4096-token context. We have not tested other model families on ARM yet.

Next step: see which server we recommend for each model size, ARM or not, in VPS for Ollama.