LLM VRAM Calculator
Estimate how much GPU memory a local language model needs. Pick a model size, quantization and context length, then see whether it fits on your GPU.
| Quantization | Weights | Total | Result |
|---|
These are estimates, not guarantees. Real usage varies by runtime (Ollama, llama.cpp, LM Studio), model file, GPU driver and batch settings. Leave 1 to 2 GB of headroom, especially if your GPU also drives a display.
How this LLM VRAM calculator works
The memory a local model needs comes from three parts: the model weights, the KV cache and runtime overhead. This calculator adds them together and compares the total with your GPU’s VRAM and your system RAM.
- Weights = parameters × bits per weight ÷ 8. A 8B model at Q4_K_M (about 4.9 bits per weight) is roughly 4.6 GB.
- KV cache = 2 × layers × KV heads × head dimension × context length × bytes per value × parallel sequences. It grows linearly with context, which is why a long context can cost several gigabytes on its own.
- Overhead covers compute buffers and the CUDA, ROCm or Metal runtime. We use a flat 0.5 GB plus 3% of the weights.
What if the model does not fit in VRAM?
Runtimes such as Ollama and llama.cpp can split a model between GPU and system RAM, a technique called partial offloading. It works, but every layer left on the CPU slows generation. The calculator estimates how many layers fit on the GPU. For a faster setup, pick a smaller model, a lower quantization or a shorter context window.
Tips to reduce VRAM use
- Use Q4_K_M as a starting point. It is a common balance between size and quality.
- Lower the context length to what you actually need.
- Quantize the KV cache to Q8_0, which roughly halves its size with a small quality cost.
- Close other GPU-heavy apps such as games and video editors.
- On Apple Silicon, memory is unified, so use the total RAM figure as your budget and leave room for macOS.
Keep learning
- How Much RAM and VRAM Do You Need for Different LLM Sizes?
- GGUF Quantization Explained: Q4_K_M vs Q5_K_M vs Q8_0
- How to Run Local LLMs With GPU Acceleration
- How to Make Local LLMs Run Faster
Frequently asked questions
How much VRAM do I need to run an 8B model?
At Q4_K_M with a 4,096 to 8,192 token context, an 8B model needs roughly 6 to 7 GB in total, so an 8 GB GPU is a comfortable fit. Use the calculator for your exact settings.
Is this calculator exact?
No. It is an estimate based on standard formulas. Different runtimes, model files and settings change real usage, so keep 1 to 2 GB of headroom.
Why does context length matter so much?
The KV cache stores attention data for every token in the context. Doubling the context doubles the KV cache, and for large models or long contexts that can rival the weights themselves.
Can I use a model with my own architecture values?
Yes. Choose “Custom model” and enter the parameter count, layer count, KV heads and head dimension from the model’s config file.
Does quantization reduce quality?
Lower-bit quantization shrinks the model but can reduce output quality, especially below 4 bits. Q4_K_M and above are popular choices for everyday use.
