Fundamentals

How much unified memory do you need for local LLMs?

By Claudio A. G. M. Filho · Updated October 21, 2026

A useful estimate separates model weights, attention cache, runtime overhead, and memory left for the system. It is a planning tool, not a loading test or a speed benchmark. Use the calculator to compare assumptions.

1. Estimate the weights

weight bytes ≈ total parameters × effective bits per weight ÷ 8

For example, 70 billion parameters at 4.5 effective bits per weight gives 39.375 billion bytes. The effective value includes quantization metadata: affine 4-bit groups of 64 with a 16-bit scale and bias work out to 4.5 bits per weight. Not every tensor is quantized alike, so compare the result with the actual model files and configuration.

2. Add the attention cache

For one sequence using conventional full attention, a planning formula is:

KV bytes = 2 × layers × KV heads × head dimension × context tokens × bytes per cache value

Use the model's KV-head count, not its total attention-head count. Context includes both the prompt and generated tokens. Batch size and concurrent sequences can increase the cache. Quantized caches also store scales and other metadata, so their size is not exactly their nominal bit depth.

This shortcut does not describe every architecture. Sliding-window or hybrid attention, multi-head latent attention, state-space models, vision encoders, and speculative decoding need architecture-specific treatment. The calculator's ordinary attention estimate must not be treated as a compatibility result for these systems.

3. Keep overhead and operating-system space separate

The calculator adds a configurable runtime allowance to weights plus cache; the starting value is 15%. This is a planning assumption, not a measured allocation. Temporary tensors, prompt processing, allocator caching, and runtime versions can change peak use.

Apple's recommendedMaxWorkingSetSize is a device-specific performance recommendation. It is not a universal 75% rule. The calculator starts with a 75% model budget and a 4 GB system reserve; it uses whichever restriction leaves less space, rather than subtracting the same reserve twice. Adjust assumptions to match the actual device and other applications. This guide does not recommend changing system wired-memory limits.

4. Read GB and GiB correctly

1 GB = 1,000,000,000 bytes; 1 GiB = 1,073,741,824 bytes. The calculator shows both for the estimate. Its capacity comparisons treat each nominal memory-tier number as GiB; this convention is explicit in the result and is not a claim that every Mac offers all of those tiers.

Worked example

Select the calculator’s Llama 3.1 70B preset with its default assumptions: 70.55B parameters, 4.5 effective bits per weight, 80 layers, 8 KV heads, head dimension 128, 8,192 tokens, 16-bit cache and a 15% runtime allowance. Weights use 39.684 GB, cache uses 2.684 GB, and the allowance adds 6.355 GB. The total is 48.724 GB ≈ 48.7 GB / 45.4 GiB, matching the calculator. The system reserve affects the capacity comparison rather than being added to this model total. This is an estimate, not a measured benchmark.

A recommended capacity tier leaves extra room within the assumed model budget. It does not guarantee that the runtime supports the model or that response speed will be useful. Test the specific setup before spending money.

Mixture-of-experts and fine-tuning

For a fully resident mixture-of-experts model, use the total parameter count, not the smaller active-per-token figure. Offloading can change residency but also changes the performance assumptions. Training and fine-tuning require additional allocations; this inference calculator does not estimate them.