Start with a task, not a tower
Running AI at home can mean private drafting, searching your own documents, coding assistance or image generation. Those jobs do not share a single hardware requirement. This guide covers candidate hardware for local inference: running an existing model. It is not a plan for training a large foundation model, and we have not measured performance on these configurations.
The memory question comes before the processor badge
Model weights, runtime overhead and working context all consume memory. Quantization can reduce model size, but the exact file and runtime matter. Longer context and concurrent requests can raise requirements. Do not buy from a parameter count alone: download or identify the actual model and check the runtime’s reported memory use. Keep room for the operating system and the rest of your work.
Check the software path first
Ollama documents supported hardware and context settings; LM Studio publishes system requirements. Check the current support for your operating system and GPU before shopping. Apple silicon, NVIDIA, AMD and Intel support varies by runtime and version. Ollama hardware support · Ollama context guidance · LM Studio requirements.
Use what you own first
Candidate: Apple silicon computer or supported Windows/Linux PC.
Install a supported local runtime and try a small model before buying hardware. Record model name, quantization, context length and your actual task. A slow but usable experiment can tell you more about your needs than a shopping list.
Check before spending: No purchase is required for this first step. A cloud service may be better value for occasional demanding jobs; check its data handling before sending private material.
A compact Mac experiment
Candidate: Mac mini M4 24GB or 32GB.
An M4 Mac mini configured with more than base memory is a candidate for someone who wants a small general-purpose desktop and a local-model experiment. Compare the complete memory configuration, not just the M4 label.
Check before spending: Unified memory is shared by the system and graphics. Installed capacity is not all available to a model; verify runtime support and model fit before ordering.
A Windows desktop graphics upgrade
Candidate: GeForce RTX 5060 Ti 16GB.
The 16 GB version is the specific candidate here. Start by checking whether the model and context you need fit into the available graphics memory, then compare the total upgrade cost.
Check before spending: NVIDIA also lists an 8 GB variant. Check the full listing, case clearance, power supply and runtime support; the same family name does not mean the same capacity.
More graphics-memory headroom
Candidate: GeForce RTX 5090 32GB.
A 32 GB RTX 5090 is a high-cost candidate for a defined workload that needs more GPU memory. Price the entire system and justify it against actual recurring use. Do not buy this tier merely to try a chatbot.
Check before spending: Memory capacity alone does not guarantee a model fits, a target speed or good training economics. Check power, cooling, physical fit and supported software first.
A Mac with more shared memory
Candidate: Mac mini M4 Pro 48GB or 64GB.
The M4 Pro Mac mini offers a higher-memory route to investigate when your model does not fit comfortably in a smaller Mac. Compare it with a dedicated-GPU desktop using the same model and task.
Check before spending: It is a different architecture and software path from an NVIDIA workstation. Do not assume CUDA-only tools work or treat shared memory and dedicated VRAM as interchangeable benchmarks.
A useful test before an expensive purchase
Choose one representative task and a known model file. Record the runtime version, quantization, context setting, time to first answer, sustained generation speed and whether the output is useful. Repeat with your normal applications open. A setup that is fast in a short demonstration may behave differently with a long document or several requests. Our buying candidates are not substitutes for those results.
Local does not automatically mean private
Check whether your application sends prompts, telemetry or search requests to external services. Keep runtimes updated and do not expose a local model server to the internet without understanding its access controls. Review the model license and the rules for any documents you use. Local processing can be useful, but its privacy properties depend on the whole workflow.
Where we would start
Try a small model on an existing supported computer. If it solves a real problem, identify the bottleneck before upgrading. Compare a higher-memory Mac for a compact general-purpose setup, or a compatible dedicated-GPU desktop when your software and graphics-memory requirements point there. Reserve the most expensive hardware for a workload and ownership budget you can explain. Our laptop guide helps when portability is essential.
Continue reading: The magazine · Tech finder · Gift ideas · Our editorial approach