Llama 3.1 8B requirements: what it really needs on a CPU server

Llama 3.1 8B requirements on a CPU server, measured: it needs 8 GB of RAM. On our two 4 GB servers it did not run at all. On 8 GB it ran with 5941 MiB of system RAM used and generated 6.3 tokens/s on a Hetzner CX33; on 16 GB it generated 9.7 tokens/s on a CX43. A GPU is optional, not required.

This matters now because 8B is the size where small open models start to feel useful, and most requirement guides you find were written for GPUs. They quote VRAM and leave CPU servers at a vague "16 GB or more". We rented five servers and checked.

Tested in the Lab Hetzner CX23, CX33, CX43, CAX11, CAX21 · Nov 2026

This page is part of our self-hosted LLM section. For the smaller sibling model, see Llama 3.2 3B requirements.

The short answer

Units used here: a vCPU is one virtual processor core on a cloud server. MiB is a mebibyte (1,048,576 bytes), so 8 GB of RAM shows up as about 7751 MiB. Tokens/s is how many tokens, the word pieces a model writes, it generates per second.

What "llama3.1" in Ollama actually is

When you run ollama run llama3.1, you get the latest tag, which Ollama's library lists as identical to llama3.1:8b. Its model page shows 8.03B parameters, quantization Q4_K_M and a 4.9GB download, with a 128K context window supported by the model.

Quantization means the weights are stored with fewer bits to save memory and disk. The tags page lists the other 8B instruct builds if you want a different trade-off, among them:

We measured only the default Q4_K_M. Smaller builds need less RAM and lose some quality; we have not tested whether q2_K or q3_K_M fit a 4 GB server.

Our measurements on five CPU servers

Setup: Ollama 0.40.14, installed natively, CPU only, on Hetzner Cloud in Helsinki with Ubuntu 24.04. One fixed prompt, a 256-token answer, temperature 0, seed 42. We report the second run for speed, so the model is already loaded. RAM is total system memory in use (from free -m) while the model was loaded, so it includes Ubuntu itself.

Measured by AI Hosting Lab on 2026-11-05 · 5 servers · method · raw data

Reading the numbers:

• CX43 (8 vCPU, 16 GB): 9.7 tokens/s generation, 447.4 tokens/s prompt reading, 7.9 s load, 6023 MiB RAM used.
• CX33 (4 vCPU, 8 GB): 6.3 tokens/s, 258.6 tokens/s prompt, 10 s load, 5941 MiB.
• CAX21 (4 ARM vCPU, 8 GB): 5.7 tokens/s, 226.5 tokens/s prompt, 12.3 s load, 5796 MiB.
• CX23 and CAX11 (2 vCPU, 4 GB): no result. Details below.

What 6.3 tokens/s feels like: a 256-token answer takes about 41 seconds. At 9.7 tokens/s it takes about 26 seconds. For background jobs that is fine; for a live chat it feels slow. Doubling the vCPUs from the CX33 to the CX43 raised speed from 6.3 to 9.7 tokens/s, about half again, not double.

Why it failed on 4 GB

This is the part most guides cannot show, because they never tried. Both 4 GB servers failed, in two different ways.

x86: Hetzner CX23, killed after about 50 seconds

Ollama checks whether a model fits before loading it. On the CX23 it reported the model would leave 1698 MiB free and started loading. About 50 seconds later the kernel's OOM killer (OOM: out of memory) stopped llama-server, with 3.5 GB of anonymous memory in use. The server had no swap. In practice your request waits, then returns an error instead of an answer.

ARM: Hetzner CAX11, failed at once

On the CAX11 the error came immediately: llama-server process has terminated: exit status 1: ggml_aligned_malloc: insufficient memory (attempted to allocate 4402.48 MB). A clean failure, at least: nothing else on the server was disturbed.

The lesson: Ollama's fit check is an estimate. On the x86 box it said yes and was wrong. Size by measured RAM, not by the estimate, and leave headroom. We have not tested the 4 GB servers with swap added.

Minimum and comfortable server

Minimum: 8 GB, 4 vCPU

Our pick is the Hetzner CX33 at 10.27 € a month: 6.3 tokens/s with 5941 MiB of 7751 MiB used, so about 1810 MiB stays free. That is enough for the model and a light API in front of it. It is not enough for n8n plus the model with real headroom. The ARM CAX21 at 12.69 € works too, slightly slower at 5.7 tokens/s.

Comfortable: 16 GB, 8 vCPU

The Hetzner CX43 at 19.35 € a month: 9.7 tokens/s, and with 6023 MiB used out of 15613 MiB it has room for n8n, a database or a second, smaller model. This is the size we suggest if the server does anything besides serving the model.

Prices, gross from the Hetzner Cloud API, Helsinki, checked 5 November 2026:

From the Hetzner Cloud API, location hel1, checked 2026-11-05. "Incl. VAT" is what our Czech (EU consumer) account pays; businesses and buyers outside the EU usually pay the price excl. VAT. Primary IPv4 is billed separately.

To size a mixed server, tick the model and your other apps in the calculator. It adds our measured RAM for each part and names the cheapest tested server that fits with 15% headroom:

Prompt length matters too

Generation speed is only half of the wait. Before the model writes anything, it reads your prompt. We measured prompt reading at 447.4 tokens/s on the CX43, 258.6 tokens/s on the CX33 and 226.5 tokens/s on the CAX21. A 1,000-token prompt, such as a pasted email thread or a document chunk for search, therefore takes about 2.2 seconds to read on the CX43 and about 3.9 seconds on the CX33 before the first word appears. For short questions this is negligible; for document workflows it adds up.

Llama 3.1 8B or a smaller model?

On the same servers, the smaller Llama 3.2 models were much faster and lighter. Here is the 8B next to Llama 3.2 3B on the two servers where both ran:

The 3B model also ran on both 4 GB servers, which the 8B could not. Pick the 8B when answer quality on open-ended questions matters more than speed, for example drafting text or answering from a longer context. Pick the 3B for classification, extraction, routing and short summaries, where it is roughly twice as fast on the same hardware and leaves RAM for other apps. Skip the 8B on CPU entirely if you need replies in a few seconds for several users at once.

Things that change the requirement

• Context length. Ollama's FAQ says it uses a 4096-token context window by default. The model supports 128K, but a longer context needs more memory. Our numbers are at the default; we have not measured long contexts yet.
• Parallel requests. Ollama processes 1 request per model at a time by default (OLLAMA_NUM_PARALLEL). More parallel requests need more memory.
• Keep-alive. By default a model unloads after 5 minutes idle. The next request pays the load time again: 7.9 s to 12.3 s on our servers.
• GPU. With a GPU, the model would sit in VRAM instead; we have not tested GPU servers yet. Our view on when that pays off is in CPU vs GPU for LLM.

Install and run it

These are the commands our benchmark used on each server:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.1:8b

Pulling the 4.9GB model took 13.6 s on the CX43, 18.6 s on the CX33 and 17.1 s on the CAX21. Then call it through the local API on port 11434, which is how our test sent its prompt.

Next step: follow our Ollama install guide to set it up on an 8 GB or 16 GB server with the settings used in these tests.