The XDA post that landed on a lot of front pages made a point most Obsidian users had quietly been testing on the side: a modern Android phone with 8 or 12 GB of RAM can run a 3-billion-parameter model at readable speed, entirely offline, and paired with Obsidian it turns the vault into something between a diary and a searchable second brain. No cloud round-trip, no per-token bill, no signal required on a plane or a subway ride.
We tested 7 Android apps that make that pairing work in practice. Every one below runs the model on the device, plugs into an Obsidian vault stored on the phone, and stays useful without a network connection.
What to look for
- On-device inference. If the app calls an API, it is not the tool we are testing.
- Reasonable models. Q4-quantised 3B or 7B is the sweet spot; anything larger hits swap and dies.
- Vault access. Direct filesystem access to the Obsidian vault beats copy-paste every time.
- Prompt speed. Under 15 tokens per second is unusable for anything longer than a sentence.
- Battery behaviour. A generation that takes two minutes at 30% CPU is fine; one that takes ten minutes at 100% is not.
- Sensible defaults. Templates, tag-aware search, and a chat history that survives an app close.
Quick comparison
| App | Best for | Free plan | Standout feature | Rating |
|---|---|---|---|---|
| Obsidian | The vault itself, on the phone | Yes | Same vault on desktop and mobile with local plugins | 4.7 stars on Aptoide |
| MLC Chat | Fastest on-device inference for chat | Yes (open source) | Vulkan-accelerated 3B and 7B models at reasonable speed | Sideload only |
| PocketPal AI | Beginner-friendly LLM runner with model library | Yes (open source) | Downloads GGUFs from Hugging Face in-app | 4.4 stars |
| Layla | Chat with local RAG over any file | Yes tier | Runs a small model over your notes without setup | 4.3 stars |
| Ollama in Termux | Full Ollama server on the phone, for power users | Yes (open source) | Same API as the desktop, so plugins just work | Sideload only |
| Smart Composer (Obsidian plugin) | Local model access from inside Obsidian mobile | Yes (open source) | Talks to an Ollama endpoint, including a local one | Free plugin |
| Text Generator (Obsidian plugin) | Templated prompts against local models | Yes (open source) | Turns any note into a prompt with variables | Free plugin |
The 7 best apps for Obsidian with a local LLM on Android
1. Obsidian — best because everything below runs against its vault
Obsidian on Android carries the same Markdown vault as the desktop, syncs through Obsidian Sync, Syncthing, or a self-hosted Git remote, and runs most plugins that do not need a browser API. Local plugins mean the AI layer stays on the phone even when the vault is elsewhere.
Where it falls short: Community plugins that assume a desktop environment fail silently. iCloud users have to lean on Obsidian Sync or Working Copy since Android does not see iCloud.
Pricing:
- Free: full app with local vaults
- Paid: Obsidian Sync (optional) for encrypted vault sync
Platforms: Android, iOS, Windows, macOS, Linux
Download: obsidian.md — also available on Aptoide and Google Play
Bottom line: Pick Obsidian first, because every AI plugin here targets a vault sitting on the phone.
2. MLC Chat — best on-device inference speed
MLC Chat ships models compiled by the MLC LLM project to run on the phone’s GPU through Vulkan. A 3B model like Phi-3.5-mini generates at 20+ tokens per second on a Snapdragon 8 Gen 3, and a 7B Q4 quant is usable on flagship phones. Chats live locally, no account needed.
Where it falls short: Sideload only, the model catalogue is smaller than PocketPal’s, and adding a custom model takes a compile step on desktop.
Pricing:
- Free: open source under Apache 2.0
- Paid: none
Platforms: Android (sideload APK)
Download: llm.mlc.ai
Bottom line: Pick MLC Chat when you want the fastest local generation on the phone and are comfortable with a sideloaded APK.
3. PocketPal AI — best on-ramp for people new to local LLMs
PocketPal AI is the friendliest way onto the local-model road. Install the app, browse a curated set of GGUF models from Hugging Face, tap to download, and start chatting. Templates cover most model families out of the box, so a first-time user is not editing chat_template JSON on a phone screen.
Where it falls short: Inference is CPU-only through llama.cpp, so speeds are lower than MLC Chat on the same hardware. Vault access is copy-paste unless you pair it with an Obsidian plugin.
Pricing:
- Free: open source under MIT
- Paid: none
Platforms: Android, iOS
Download: Google Play
Bottom line: Pick PocketPal if this is the first time you are running a model on a phone.
4. Layla — best when the chat needs to see your notes
Layla runs small models locally and layers a light retrieval-augmented step on top, so a prompt can reference notes or PDFs you point it at. For an Obsidian user, this means “summarise last week’s daily notes” works without any plugin work, as long as the vault lives in a folder Layla can read.
Where it falls short: RAG quality on a small model is uneven. The free tier caps model size; the paid tier removes it. UI polish trails PocketPal.
Pricing:
- Free tier with limited models
- Paid: one-time upgrade unlocks larger models
Platforms: Android, iOS
Download: Google Play
Bottom line: Pick Layla if the goal is “answer questions about my notes” and you do not want to build the pipeline yourself.
5. Ollama in Termux — best power-user setup that mirrors the desktop
Installing Ollama inside Termux gives the phone the same HTTP server the desktop runs, on the loopback interface. Any Obsidian plugin that speaks Ollama then talks to it directly. Battery is heavier because Termux does not exit gracefully, but performance beats most GUI runners on the same phone.
Where it falls short: Termux from the Play Store is out of date; sideload from F-Droid or the official GitHub build. Foreground service permissions need care, and thermal throttling can hurt long generations.
Pricing:
- Free: open source
- Paid: none
Platforms: Android (Termux)
Download: ollama.com with Termux
Bottom line: Pick this if you are already comfortable in Termux and want the desktop Ollama experience on the phone.
6. Smart Composer — best Obsidian plugin for pointing at a local model
Smart Composer is the community plugin that turns a chat inside Obsidian into a conversation with any OpenAI-compatible endpoint, including local Ollama or LM Studio running on the same phone through Termux, or on a home server across the LAN. Prompts can @-mention notes, folders, and tags, so a “rewrite this paragraph in the tone of last month’s daily notes” prompt has real context.
Where it falls short: The plugin still assumes you can reach a server, so if the phone is offline and there is no local server running it does nothing.
Pricing:
- Free: open source under MIT
- Paid: none
Platforms: Android, iOS, Windows, macOS, Linux (inside Obsidian)
Download: github.com/glowingjade/obsidian-smart-composer
Bottom line: Pick Smart Composer as the bridge between Obsidian and the model, whether that model runs on the phone or on your home server.
7. Text Generator — best plugin for prompt templates
Text Generator treats notes as templates. Wrap a variable, add a system prompt, and running the template on any note fills the placeholders and sends the result to the configured model. It works against local Ollama, LM Studio, or hosted providers, and the template library on the community forum is large.
Where it falls short: The syntax takes an afternoon to learn. Debugging a failed template on mobile is more painful than on desktop.
Pricing:
- Free: open source under GPLv3
- Paid: none
Platforms: Android, iOS, Windows, macOS, Linux (inside Obsidian)
Download: text-gen.com
Bottom line: Pick Text Generator if you want reusable prompt templates that turn a note into a repeatable AI action.
How to pick
If you do not yet have a vault on the phone, install Obsidian first and sync it.
If you want the fastest local generation and are fine with a sideloaded APK, run MLC Chat.
If this is your first time trying local models on a phone, install PocketPal AI and pick a 3B model.
If the chat should see your notes with no manual pipeline, try Layla with the vault folder shared.
If you already live in Termux, install Ollama there and let every Obsidian plugin point at localhost.
To bring the model into Obsidian itself, add Smart Composer as the chat surface.
For repeatable templated prompts on notes, add Text Generator as well.
FAQ
What is the smallest phone that can run a useful model?
An 8 GB RAM phone from the last two flagship generations will run a 3B model at usable speed. A 12 GB RAM phone opens up 7B Q4 quants. Below 6 GB, the OS keeps swapping the model out and speeds fall off a cliff.
Which model should I start with?
Phi-3.5-mini and Llama-3.2-3B are both good defaults for a phone. Both fit under 3 GB in Q4, generate coherently, and know enough general knowledge to be useful. Step up to a 7B like Qwen2.5-7B or Mistral-7B on a flagship device.
Will running the model wreck my battery?
Short generations of a couple of hundred tokens are fine. Long agent runs, RAG pipelines that re-encode a big vault, or continuous foreground use will drain a normal phone in a couple of hours. Treat the local model like a short-form assistant, not a background service.
Can Obsidian mobile call a model running on my desktop?
Yes. Point Smart Composer at your desktop’s local IP and Ollama port over your home network. Add Tailscale or WireGuard if you want the same connection to work when you are away from home. The phone stays private; the model is just somewhere else.
