How Much VRAM and RAM Do You Need to Run LLMs Locally? | GB By Model

A 7B model needs roughly 14 GB of VRAM at full precision, 7 GB at 8-bit, and 3.5 GB at 4-bit — before the context cache and runtime overhead.

Skip the arithmetic and you end up with either a card you can’t use or one that dies mid-prompt. Every estimate for how much VRAM and RAM you need to run LLMs locally boils down to two multipliers: how many parameters the model carries, and how many bytes each parameter takes in memory.

Weight memory is the floor, not the total. The runtime, the operating system, temporary tensors, and the KV cache stack on top, and the cache grows with context length and batch size. Leave headroom rather than matching the model size exactly.

VRAM You Need For Local LLMs: The Weight Math

Each parameter costs about 2 bytes in FP16 or BF16, 1 byte at 8-bit, and roughly half a byte at 4-bit. Hugging Face’s quantization documentation describes 8-bit loading as roughly halving memory use and 4-bit loading as a roughly fourfold reduction.

A 70B model at 4-bit still wants about 35 GB before anything else loads, which is why that size lands on multiple GPUs, unified memory, or CPU offload. Quantization metadata and runtime buffers push the real figure higher, so treat these as lower bounds.

In code, 4-bit loading is a BitsAndBytesConfig(load_in_4bit=True) passed to from_pretrained with device_map="auto". Hugging Face recommends confirming the true allocation with model.get_memory_footprint() rather than trusting the math.

Download size is a poor guide to runtime fit. A 4-bit checkpoint of a 13B model lands near 7 GB on disk; a long context and a generous batch push the loaded footprint past 8 GB.

Do You Need 16 GB Or 24 GB Of VRAM?

Sixteen gigabytes handles 7B to 13B models at 4-bit with room for a moderate context. Twenty-four gigabytes buys 13B to 34B quantized models, or the same 13B at a far longer prompt.

• 8 GB: 3B to 7B models at 4-bit, short to moderate contexts.
• 12 GB: 7B to 13B at 4-bit, depending on context length and runtime.
• 16 GB: 7B to 13B at 4-bit, comfortably. Hugging Face documents fine-tuning Llama 13B on a 16 GB NVIDIA T4 with nested quantization, sequence length 1,024, batch size 1, and four gradient-accumulation steps.
• 24 GB: 13B to 34B quantized, or a 13B model at long context.
• 48 GB and up: 34B-class quantized models reliably, plus some 70B deployments split across multiple GPUs or offloaded to CPU.

Laptop cards cap out lower than their desktop namesakes. An RTX 4090 laptop GPU carries 16 GB of VRAM, the 4080 laptop carries 12 GB, and the 4070 laptop sits at 8 GB.

If the card is what you’re choosing, this roundup of tested laptops for running local LLMs lines up each price tier with the VRAM it actually ships.

System RAM, Apple Silicon, And Hardware Gates

Sixteen gigabytes of system RAM is a workable floor for small quantized models, 32 GB is the better general target, and 64 GB or more covers large models, CPU inference, model conversion, and offloading.

System RAM does not replace VRAM for GPU execution. Runtimes can park part of the model in CPU memory, and Hugging Face’s bitsandbytes quantization guide shows how to control that split with device_map="auto" and explicit max_memory limits. The same page notes that 8-bit CPU-offloaded weights are stored in float32 on the CPU, so offloading returns far less memory than the label suggests.

Apple Silicon Macs pool unified memory between macOS, open apps, and the model, so a 16 GB machine has well under 16 GB left for weights. Integrated graphics and non-NVIDIA backends need a different runtime, because a file’s 4-bit label says nothing about whether your machine can load it.

Context length is the variable people forget. The KV cache holds attention state for every token in the prompt and every token generated, so doubling context roughly doubles that slice of memory, and serving two people at once doubles it again. Long system prompts and retrieved documents pay for themselves in VRAM.

Training is a separate problem. Gradients, optimizer states, and activations push memory needs well past inference, so a card that runs a quantized 7B model will not full fine-tune it.

Work backwards from the model you actually intend to run: 4-bit weights for a 13B model take about 6.5 GB, so a 12 to 16 GB card with 32 GB of system RAM holds it with context to spare. Move up to 34B or 70B and the GPU count, the RAM, or both have to move with it.