The XDA writeup last week ended where every home-lab thread ends: only one of seven local LLMs held up as the brains of a smart home when the automations got interesting. The rest either hallucinated on tool calls, missed the “turn off the porch light” intent, or ran so hot the NAS fans spun for hours. A local model on a NAS is a legitimate answer to voice control, but the choice of runtime matters more than the choice of model.
We tested 7 desktop apps that run on a home NAS or a mini PC on the same LAN, expose an OpenAI-compatible endpoint, and integrate cleanly with Home Assistant’s LLM-backed voice pipeline. Everything below runs offline once the weights are downloaded, and everything below has a documented Docker image.
What to look for
- OpenAI-compatible API. Home Assistant’s Assist expects it, and every other tool below plugs in the same way.
- Tool-call support. A model that cannot emit a structured function call cannot flip a light.
- GGUF or GPTQ model support. If the NAS has no GPU, a Q4 GGUF on CPU is the honest starting point.
- Sensible memory footprint. A 7B Q4 model should stay under 6 GB of RAM at rest.
- Warm start. Cold-loading a 7B model on every voice command is a five-second lag you notice.
- A stable process manager. NAS reboots weekly for updates; the model should come back automatically.
Quick comparison
| App | Best for | Platforms | Free plan | Standout feature |
|---|---|---|---|---|
| Ollama | The default runner most Home Assistant guides target | Linux, macOS, Windows | Yes (open source) | Model catalogue and a stable OpenAI-compatible endpoint |
| LocalAI | All-in-one drop-in for OpenAI APIs, including audio | Linux, macOS, Windows (Docker) | Yes (open source) | Handles chat, embeddings, TTS, and STT from one container |
| LM Studio | GUI on the NAS host and a server for the LAN | Windows, macOS, Linux | Yes | Model discovery and one-click server toggle |
| Open WebUI | Web chat frontend that also talks to Home Assistant | Linux, macOS, Windows (Docker) | Yes (open source) | Multi-user chat with Ollama, LocalAI, or any OpenAI-compatible backend |
| Home Assistant | The consumer of the endpoint | Windows, macOS, Linux (HAOS, Docker) | Yes | Native Assist LLM integration with tool-call routing |
| vLLM | Best throughput on a NAS with a GPU | Linux (Docker) | Yes (open source) | Continuous batching and paged attention for multi-user use |
| llama.cpp | The bare-metal C++ runner behind most of the above | Linux, macOS, Windows | Yes (open source) | Runs GGUF on CPU on almost any hardware |
The 7 best apps for a NAS smart-home LLM
1. Ollama — best default runner and the one most guides target
Ollama wraps llama.cpp with a clean CLI, a model library, and an OpenAI-compatible HTTP server that Home Assistant’s Assist LLM integration talks to out of the box. Install it on the NAS, ollama pull qwen2.5:7b-instruct, and point Home Assistant at http://<nas-ip>:11434. The service manager keeps it warm across reboots.
Where it falls short: Multi-user concurrency is single-lane; parallel requests queue rather than batch. Windows support arrived recently and lags Linux on stability.
Pricing:
- Free: open source under MIT
- Paid: none for self-hosting
Platforms: Linux, macOS, Windows (native and Docker)
Download: ollama.com
Bottom line: Pick Ollama first, and only leave when a specific limitation pushes you away.
2. LocalAI — best if you want chat, speech, and embeddings from one container
LocalAI is the drop-in that covers the whole OpenAI surface: chat completions, embeddings, TTS, STT, and image generation, all local. Home Assistant’s Wyoming protocol can point at LocalAI’s Whisper endpoint for STT and Piper for TTS, meaning the entire voice loop lives on the NAS.
Where it falls short: The config file is dense and needs a first-time read. GPU acceleration paths (CUDA, ROCm, Vulkan) are per-build and picking the wrong image gives silent CPU fallback.
Pricing:
- Free: open source under MIT
- Paid: none
Platforms: Linux, macOS, Windows (Docker)
Download: localai.io
Bottom line: Pick LocalAI if the goal is a self-contained voice stack with STT, TTS, and chat under one roof.
3. LM Studio — best when a friendlier UI matters
LM Studio started as a desktop chat app and grew a headless server mode. On a NAS with a display or a remote KVM session, the model picker and quantisation chooser are more forgiving than a raw CLI. The built-in server exposes the same OpenAI-compatible endpoint Home Assistant expects.
Where it falls short: The core app is closed source, which is a hard no for some home-lab setups. Headless mode is available but still a second-class citizen.
Pricing:
- Free: full app
- Paid: none
Platforms: Windows, macOS, Linux
Download: lmstudio.ai
Bottom line: Pick LM Studio if you want a UI to try models before locking one in for the smart-home role.
4. Open WebUI — best chat frontend that keeps humans in the loop
Open WebUI is the browser-based ChatGPT-style frontend that runs against Ollama, LocalAI, or any OpenAI-compatible endpoint. On a NAS, it doubles as a testing surface: run a prompt against the same model the Assist pipeline uses, tweak the system prompt, then update Home Assistant. Multi-user support with roles keeps the family off the admin console.
Where it falls short: It is a chat UI, not part of the automation path itself. Setup requires a persistent volume and a reverse proxy for LAN or Tailscale access.
Pricing:
- Free: open source under MIT
- Paid: none
Platforms: Linux, macOS, Windows (Docker)
Download: openwebui.com
Bottom line: Pick Open WebUI as the human-facing chat on top of whichever runtime you settle on.
5. Home Assistant — best consumer of the endpoint
Home Assistant’s Assist LLM integration is what turns “kitchen light on” into a tool call. Point it at Ollama, LocalAI, LM Studio, or any OpenAI-compatible endpoint, tick “expose to Assist” on the entities you want the model to control, and voice runs entirely on the LAN. The prompt template is editable, which is how you keep the model from hallucinating rooms.
Where it falls short: Tool-call quality depends on the model, and Home Assistant will not tell you when the model returns a plausible but wrong entity id. Debugging lives in the Assist debug pane and takes a while to master.
Pricing:
- Free: open source
- Paid: optional Nabu Casa Cloud, not required for local voice
Platforms: Windows, macOS, Linux (HAOS, Docker, VM)
Download: home-assistant.io
Bottom line: Pick Home Assistant because the LLM is only interesting if something acts on its output, and this is the something.
6. vLLM — best if the NAS has a real GPU
vLLM is the throughput-focused runtime. Continuous batching, paged attention, and a proper multi-request queue turn a small NVIDIA card into a family-scale endpoint that can serve voice requests, an Open WebUI tab, and an Obsidian plugin at the same time without one blocking the others.
Where it falls short: GPU only, so a CPU-only NAS is out. Model compatibility is narrower than Ollama’s and adding a new architecture is a build step.
Pricing:
- Free: open source under Apache 2.0
- Paid: none
Platforms: Linux (Docker)
Download: github.com/vllm-project/vllm
Bottom line: Pick vLLM if the NAS has a supported NVIDIA card and multiple family members will be hitting the model at once.
7. llama.cpp — best bare-metal runner for constrained hardware
llama.cpp is the C++ project that powers most of the runtimes above. Running it directly, with the built-in HTTP server, is the leanest way to squeeze a 7B model onto a Synology or an old ARM box. Support for Metal on Apple Silicon and CUDA on NVIDIA lets one binary cover most home setups.
Where it falls short: No model management, no auto-updates, no service scripts out of the box. It is a toolkit, not a product.
Pricing:
- Free: open source under MIT
- Paid: none
Platforms: Linux, macOS, Windows (source, binary, or Docker)
Download: github.com/ggerganov/llama.cpp
Bottom line: Pick llama.cpp if the NAS is small, the OS is unusual, and you want the least abstraction possible over the model file.
How to pick
If you are starting from scratch, install Ollama on the NAS and point Home Assistant at it.
If the voice stack needs STT and TTS on the same box, swap Ollama for LocalAI and pull Whisper and Piper images.
If you want a UI to compare models before committing, install LM Studio and use its server mode.
If the family wants to chat with the same model that runs the automations, add Open WebUI on top.
If the NAS has an NVIDIA GPU and more than one heavy user, switch to vLLM for concurrency.
If the hardware is unusually small and every megabyte matters, run llama.cpp directly.
FAQ
Can I really run a useful model on a NAS?
Yes. A 7B Q4 model in llama.cpp on a modern eight-core CPU generates at readable speed for short voice-command responses. If the NAS has 16 GB of RAM and an NVMe cache, the model loads in seconds after a warm start. For long chats or reasoning, add a GPU.
Which model handles Home Assistant tool calls best?
Qwen 2.5 Instruct, Llama 3.1 8B Instruct, and Hermes-3-Llama are the reliable local options at the 7B-8B size. Below 7B, tool-call formatting starts to break. Test with the Assist debug pane on a handful of realistic commands before committing.
Does the model see my whole house state?
Only the entities you explicitly expose to Assist. The exposure toggle is per-entity, so a light in the office does not have to leak into the model’s context. Keep the exposed set small; the prompt is cheaper and the model makes fewer mistakes.
What happens when the internet drops?
Nothing changes. STT, LLM, and TTS all run on the NAS, so voice control keeps working. The only path that needs WAN is remote access, and Nabu Casa or a self-hosted VPN both fix that separately.
