The XDA piece on pairing local cameras with Home Assistant’s AI hit a nerve for a reason. The default of most smart cameras is still “the vendor sees the footage first”. Home Assistant plus Frigate plus an LLM inside a local Ollama flips that: raw RTSP goes to a local server, object detection runs on a Coral or GPU, and the LLM only touches the frames the object detector already flagged. Nothing leaves the house.
The pieces are open source and moving fast. Seven desktop-side apps to make Home Assistant AI camera vision actually work, ranked so the first is where every install starts and the last is where it stops mattering.
What to look for in a local camera AI stack
The pipeline splits into four jobs:
- Ingest RTSP or ONVIF streams without pulling them through a vendor cloud.
- Object detection at frame rate on cheap hardware. Coral Edge TPU for USB accelerators, Frigate for the software layer.
- Scene analysis with an LLM that can describe what’s in a bounding box in natural language (“a package on the porch”, “a raccoon in the trash”).
- Event routing into Home Assistant so notifications, automations, and dashboards can react.
Everything below runs entirely on hardware you own, on a Windows, macOS, or Linux desktop or homelab node.
Quick comparison
| App | Best for | License | Hardware floor | Standout |
|---|---|---|---|---|
| Home Assistant | The hub every other tool plugs into | Apache 2.0 | 2 GB RAM | Best automation router on this list |
| Frigate | Local object detection at frame rate | MIT | 4 GB RAM, Coral or GPU | Fastest local NVR that speaks MQTT |
| LLM Vision | Natural-language descriptions of camera events | MIT | Uses your LLM’s hardware | Turns bounding boxes into readable sentences |
| Scrypted | Camera ingest and Apple HomeKit bridging | MIT | 2 GB RAM | Best way to expose non-HomeKit cams to HomeKit |
| Blue Iris | Windows-native NVR with mature UI | Proprietary | Windows PC | The Windows heavyweight; rock-solid recording |
| MQTT Explorer | Debugging Frigate → Home Assistant messages | MIT | 200 MB RAM | Reads any of the traffic Frigate emits |
| Ollama | Local LLM that LLM Vision talks to | MIT | 8 GB RAM + GPU helpful | Where the multimodal model actually runs |
The apps
1. Home Assistant – Best hub
Home Assistant is the hub every camera integration in this stack points at. It handles the entity model, automations that trigger on Frigate events, dashboards that show the last snapshot, notifications to phones, and the settings UI for LLM Vision. Even if you already run Blue Iris or Scrypted for the recording side, Home Assistant is where the “raccoon in the trash → light on → notification” logic lives.
Where it falls short: HAOS is opinionated about owning its host. In Docker you lose supervisor add-ons. Full learning curve past the basics.
Pricing:
- Free, Apache 2.0
- Nabu Casa Cloud at $6.50/month adds remote access without exposing a port
Platforms: Linux (HAOS or Docker), Windows or macOS via VM/Docker Desktop.
Download: Home Assistant
Bottom line: Install first. Every other app on this list assumes it’s there.
2. Frigate – Best local NVR and object detection
Frigate is the piece that turns raw RTSP into “person at the door”, “car in the driveway”, or “package delivered”. It pulls streams from your cameras, runs detection on a Coral Edge TPU or GPU, publishes events over MQTT for Home Assistant to consume, and records clips only for the events you care about. The 2026 release added a native LLM plug that hands snapshots straight to a multimodal model.
Where it falls short: RAM usage climbs with camera count. Coral TPU is officially recommended and not always in stock. Configuration YAML is long the first time.
Pricing:
- Free, MIT-licensed
- Frigate+ paid trained-model add-on at $50/year is optional
Platforms: Linux (Docker), best on x86_64 with a Coral USB accelerator or an nVidia GPU.
Download: Frigate NVR
Bottom line: Install second. It’s the reason “AI cameras without the cloud” is a real sentence.
3. LLM Vision – Best for readable event descriptions
LLM Vision is the Home Assistant integration that hands camera snapshots and clips to a multimodal LLM and returns a natural-language description. It works with Ollama, OpenAI, Anthropic, Google Gemini, and any OpenAI-compatible endpoint. In practice: Frigate flags “person” at the front door, LLM Vision asks Ollama “describe what this person is doing”, and the summary lands as a phone notification.
Where it falls short: Descriptions are only as good as the model behind them. Small vision models occasionally hallucinate. Latency on a home GPU is noticeable.
Pricing:
- Free, MIT
Platforms: Runs inside Home Assistant; the underlying LLM lives wherever you host it (Ollama on Linux, LM Studio, OpenAI in cloud).
Download: LLM Vision on GitHub
Bottom line: Install after Frigate. This is what turns bounding boxes into “someone left a bag by the door”.
4. Scrypted – Best camera ingestion and HomeKit bridge
Scrypted is the swiss army knife for cameras that don’t speak the protocol you want. It pulls in RTSP, ONVIF, Ring, Nest, Reolink, Amcrest, and re-exposes them as HomeKit Secure Video, RTSP, or WebRTC. On a household with a mix of “cheap cam that speaks RTSP” and “iPhone user who wants HomeKit”, Scrypted is the translator.
Where it falls short: Not itself an object detector at the same quality as Frigate. Two tools running in parallel is common.
Pricing:
- Free, MIT
- Optional NVR plugin bundled subscription for cloud sync
Platforms: Linux (Docker), Windows, macOS.
Download: Scrypted
Bottom line: Install when the household mixes cameras from multiple ecosystems.
5. Blue Iris – Best Windows-native NVR
Blue Iris is the veteran. Runs on a Windows PC, records to local disk, integrates with Home Assistant over MQTT, and its dashboard is the most polished in the space. Pair with Frigate for AI or with its own DeepStack/CodeProject.AI integration for object detection.
Where it falls short: Windows-only. Proprietary. UI is 2010s Windows in spirit.
Pricing:
- One-time license around $70 (LE) to $130 (Full)
- Free trial available
Platforms: Windows.
Download: Blue Iris
Bottom line: Best pick when a Windows machine is already the always-on home server.
6. MQTT Explorer – Best debugging tool for the pipeline
MQTT Explorer is the tool you open when Frigate is emitting events, Home Assistant isn’t reacting, and you need to see the traffic between them. It subscribes to any topic tree, shows history in a live feed, and lets you republish messages to test automations. Once the pipeline is stable you don’t touch it; while you’re setting it up it’s essential.
Where it falls short: Not really an “app” you run daily. Source-available rather than fully open source.
Pricing:
- Free
Platforms: Windows, macOS, Linux.
Download: MQTT Explorer
Bottom line: Install once, use for a week while wiring things up, forget you have it.
7. Ollama – Best local LLM to back LLM Vision
Ollama is the runtime the local multimodal model actually lives on. LLaVA and its descendants, Qwen2.5-VL, moondream, and other small vision models pull in a couple of commands and run happily on a GPU or Apple Silicon. LLM Vision points at the Ollama endpoint and every event snapshot round-trips through it.
Where it falls short: GPU or Apple Silicon strongly recommended for speed. CPU-only is possible but slow. Model choice affects description quality.
Pricing:
- Free, MIT
Platforms: Linux, macOS, Windows.
Download: Ollama
Bottom line: Install as the engine behind LLM Vision when you want everything local.
How to pick the right one
If you’re starting from scratch: Home Assistant first, Frigate second, LLM Vision third with a small vision model on Ollama. That is the reference stack.
If Apple HomeKit matters: add Scrypted to expose non-HomeKit cameras.
If the household server is already a Windows box: Blue Iris takes over the NVR role and hands events to Home Assistant.
When Frigate isn’t triggering the automation you expect: MQTT Explorer to see what’s actually being published.
If cloud LLMs are acceptable: skip Ollama and point LLM Vision at OpenAI or Gemini for higher description quality at the cost of leaving the house.
FAQ
Can Home Assistant do camera AI without Frigate?
There are lighter integrations (motion detection through the camera integration itself, DeepStack, CodeProject.AI). Frigate is where the current best-in-class local object detection lives. Most people end up on it.
Do I need a Coral Edge TPU?
Strongly recommended. Frigate runs on a CPU but at a fraction of the frame rate. A single Coral USB stick handles 3 to 5 cameras comfortably. A GPU works too, at higher power cost.
Which LLM is best for LLM Vision?
For local, Qwen2.5-VL and moondream are the current front runners on modest hardware. For cloud, GPT-5 or Gemini-2.5-Pro produce noticeably better descriptions at the cost of privacy.
Can I run this without cloud at all?
Yes. Home Assistant + Frigate + LLM Vision + Ollama covers the entire pipeline locally. Only phone notifications leave the house, and even those can be routed through a self-hosted Ntfy or Gotify server.
Does this work with wireless cameras?
Any camera that emits RTSP or supports ONVIF works. Wi-Fi cameras are fine; a stable network matters more than a wired camera.
