That old gaming PC sitting in the closet has a lot of life left. A 2018-era Ryzen or Intel machine with 32 GB of RAM and a used GPU will run a 7B model at conversational speed and hold a 13B model comfortably. Point the family laptops at it and there is a private, always-on AI that answers on the LAN in a couple of seconds. The best apps for turning an old PC into a local AI server pick their models well, ship an HTTP API for other tools, and let the machine sleep between queries. We picked seven that turn ten-year-old hardware into a usable AI service.
What to look for in a local AI server app
Six things matter for old-hardware AI:
- Backend and format.
llama.cpp(GGUF) is the widest-supporting; MLC and ONNX are the alternatives. - CPU-only path. Whether the app runs at usable speed on a machine without a GPU.
- GPU support breadth. Nvidia (CUDA), AMD (ROCm), Intel (SYCL), and Apple Silicon (Metal).
- Concurrent request handling. Whether the family can query at the same time without one starving the other.
- OpenAI-compatible API. Whether other tools can point at the server and think they are talking to the OpenAI API.
- Memory ceiling. RAM plus VRAM decides which model fits. Q4 quantization roughly halves the memory need.
Quick comparison
| App | Best for | GPU support | API | Standout |
|---|---|---|---|---|
| Ollama | Simplest server | Nvidia, AMD, Apple | OpenAI-compatible | One command runs a model |
| LM Studio | Friendly UI | Nvidia, AMD, Apple | OpenAI-compatible | Best model discovery |
| Jan | Fully local ChatGPT | Nvidia, AMD, Apple | OpenAI-compatible | Chat plus server in one app |
| GPT4All | CPU-only baseline | Nvidia, AMD, CPU | Local API | Runs on ancient hardware |
| llama.cpp | Bare metal control | Nvidia, AMD, Apple, CPU | Bare HTTP | The engine under most of the list |
| Open WebUI | Web frontend for Ollama | Any (client) | Talks to Ollama | ChatGPT-shaped UI for the LAN |
| LocalAI | Drop-in OpenAI replacement | Nvidia, AMD, CPU | OpenAI-compatible | Serves images, audio, embeddings too |
The apps
1. Ollama, best “run a model in one command”
Ollama ships a background service that pulls, quantizes, and serves models from a curated library. One command (ollama run llama3.2) downloads the weights, loads the model, and drops the user into a chat. The service exposes an OpenAI-compatible endpoint at localhost:11434, so any tool that talks to OpenAI can point at the old PC and get the same behaviour.
Where it falls short: Ollama’s own library uses its custom manifest format. Bringing a random GGUF from Hugging Face requires a small Modelfile and an import step.
Pricing: Free, MIT-licensed.
Platforms: Windows, macOS, Linux.
Download: Ollama
Bottom line: Start here. Install it, run one command, and the LAN has an AI server.
2. LM Studio, best friendly UI
LM Studio is a desktop app that browses Hugging Face, downloads GGUF models, and runs them locally. It has a chat interface and a “local server” toggle that exposes an OpenAI-compatible endpoint. The UI shows RAM and VRAM predicted use before download, which is genuinely useful when picking a quantization.
Where it falls short: Closed-source. Some parts of the app rely on internet reachability even for local use, though the model runs offline.
Pricing: Free for personal use.
Platforms: Windows, macOS, Linux.
Download: LM Studio
Bottom line: Pick this when the friend running the old PC does not want to touch a terminal.
3. Jan, best fully local ChatGPT clone
Jan is a desktop app that ships as a ChatGPT-shaped client, a model manager, and a local API server in one bundle. Model downloads are visible, and the “server mode” makes the same models available to other apps on the LAN. The whole design is open-source and privacy-first.
Where it falls short: Newer than Ollama and LM Studio; occasional rough edges on model support.
Pricing: Free, AGPL-licensed.
Platforms: Windows, macOS, Linux.
Download: Jan
Bottom line: The pick if the goal is a single app that acts as both chat client and server on the LAN.
4. GPT4All, best CPU-only baseline
GPT4All was built for the case where there is no GPU at all. It runs GGUF models on the CPU with usable performance on any machine from 2018 onward. The desktop app manages models, and a local API server can be turned on from settings.
Where it falls short: Speed on CPU is limited by memory bandwidth. Expect 4 to 8 tok/s on a 7B model on a modern CPU without a GPU.
Pricing: Free, MIT-licensed.
Platforms: Windows, macOS, Linux.
Download: GPT4All
Bottom line: For the office PC with no GPU. It still runs an assistant.
5. llama.cpp, best bare-metal control
llama.cpp is the engine that most of the apps above wrap. Running it directly is the fastest path on old hardware because there is no overhead. The llama-server binary exposes an HTTP endpoint; the llama-cli binary runs interactive chat. Everything is one C++ project with no runtime.
Where it falls short: Command-line only. Configuration lives in flags, not a UI.
Pricing: Free, MIT-licensed.
Platforms: Windows, macOS, Linux (source builds everywhere).
Download: GitHub
Bottom line: The pick when the goal is to squeeze every last token per second out of the old machine.
6. Open WebUI, best web frontend
Open WebUI is a self-hosted web UI that talks to Ollama (or any OpenAI-compatible endpoint). Install it on the old PC, and every device on the LAN gets a ChatGPT-like page at http://server-ip:3000. Users, permissions, RAG documents, and multi-model routing are all built in.
Where it falls short: Runs on top of a model backend, so needs Ollama or an OpenAI-compatible server behind it.
Pricing: Free, self-hosted; BSD-3-Clause.
Platforms: Docker on any host.
Download: GitHub
Bottom line: The user interface most families want. Install it right after Ollama.
7. LocalAI, best OpenAI drop-in
LocalAI replaces the OpenAI API on the LAN. Chat completions, embeddings, image generation (Stable Diffusion), text-to-speech, and speech-to-text all live behind the same HTTP endpoints OpenAI uses. Any tool with an OpenAI SDK works against LocalAI by changing one env var.
Where it falls short: Broad scope means slower startup. Each modality has its own model choices.
Pricing: Free, MIT-licensed.
Platforms: Docker on Linux, Windows (WSL2), macOS.
Bottom line: Pick this when the goal is to swap OpenAI for a local server across many apps at once.
How to pick the right combination
For a home-lab setup that just needs to work: Ollama plus Open WebUI. One serves models on the LAN; the other gives the family a page to visit.
For a friend who will not touch a terminal: LM Studio on the desktop and let them chat locally.
For an old CPU-only office box: GPT4All with a 3B model. Or llama.cpp with the same model if a terminal is fine.
For an all-in-one that gives both chat and server without splitting into two apps: Jan.
For a household running many AI tools (personal assistants, coding IDEs, note apps): LocalAI so every tool sees an OpenAI-shaped endpoint.
FAQ
Which model should I run first? Llama 3.2 3B or Qwen 2.5 3B at Q4_K_M. Both fit in 4 GB of RAM, run at 15 to 30 tok/s on a modern CPU alone, and give near-flagship answers for common questions.
Do I need a GPU? No, but it changes what you can run. Without a GPU, expect 7B models at 5 to 10 tok/s. With even a used Nvidia RTX 3060 (12 GB), a 13B model at 30 to 50 tok/s becomes practical.
How much RAM do I need? 16 GB is the floor for 7B models. 32 GB opens 13B territory. 64 GB or more starts to hold 34B and 70B (with quantization).
Can I access this from outside the house? Yes, over a mesh VPN like Tailscale. Do not expose an Ollama or LocalAI endpoint on the public internet. There is no built-in auth by default.
How much power does an old PC pull at idle? An older desktop with a GPU idles at 40 to 80 W. That is $50 to $100 a year of electricity in most markets. If the machine is only queried a few times a day, set BIOS wake-on-LAN and put it to sleep between sessions.
How does this compare to running Ollama on a Mac mini? An Apple Silicon Mac mini with 16 or 24 GB of unified memory is the most power-efficient path. If the old PC still has value elsewhere, keep it. If starting fresh, a used M1 mini often beats a 2018 tower on watts per token.
