Training a local model on its own failures is not quite pretraining, but done right it teaches a small model to stop repeating the mistake that annoyed you yesterday. That is the promise behind the “self-improvement” loop the local LLM community has been iterating on all year: log the model’s wrong answers, curate them into a small feedback dataset, run a light LoRA fine-tune, and watch the error rate drop. These are the best apps for local LLM self-improvement from failures on desktop, whether you are running a 7B model on a laptop or a 70B on a workstation.
What to look for in a self-improvement stack
- Feedback capture built in. A tool that logs prompt, response, and a thumbs-up/thumbs-down signal (or better, a corrected answer) is where the loop starts.
- Dataset curation. The failure log has to become a clean instruction-tuning dataset before it is useful.
- Efficient fine-tuning. LoRA and QLoRA cut a 24 GB fine-tune to something a single consumer GPU can run overnight.
- Local inference for evaluation. The same tool that serves the model to your agent should serve the tuned version so you can A/B the two.
- Programmatic prompt optimisation. For a lot of failure modes the fix is not fine-tuning but a better prompt. Optimisation frameworks make that mechanical.
Quick comparison
| App | Best for | Platforms | Free plan | Starting price/mo | Rating |
|---|---|---|---|---|---|
| LangChain | Chaining feedback capture into a fine-tune workflow | Windows, macOS, Linux | Full | Free (open source) | 4.5 |
| LM Studio | Everyday inference plus dataset export | Windows, macOS, Linux | Full | Free | 4.6 |
| Ollama | Serving fine-tuned models to any client | Windows, macOS, Linux | Full | Free (open source) | 4.7 |
| Axolotl | Reproducible YAML-driven LoRA runs | Linux (Windows via WSL, macOS via container) | Full | Free (open source) | 4.6 |
| Unsloth | 2-5x faster LoRA on a single GPU | Windows, Linux | Full | Free tier, Pro available | 4.8 |
| Text Generation WebUI | Local UI for inference, evaluation, and manual labelling | Windows, macOS, Linux | Full | Free (open source) | 4.4 |
| DSPy | Optimising prompts and pipelines from labelled failures | Windows, macOS, Linux | Full | Free (open source) | 4.7 |
The apps
1. LangChain – best for chaining the feedback loop
LangChain is not a fine-tuner. What it does well is give you the building blocks to capture every prompt-response pair from an agent, tag the failures, and route them into a training dataset. The Callbacks and Tracing layers are exactly what a self-improvement loop needs: a durable record of what the model said, what the user did with the answer, and how it was corrected.
Where it falls short: The API surface is large and changes often. Older code samples on the internet are frequently wrong.
Pricing:
- Free: The framework.
- Paid: LangSmith is a paid managed tracer if you want a hosted dashboard.
Platforms: Windows, macOS, Linux.
Download: langchain.com — Source (GitHub)
Bottom line: Start here for the plumbing between the model, the app, and the training data.
2. LM Studio – best for daily-driver inference and dataset export
LM Studio is the easiest local inference UI on desktop. Point it at a Hugging Face model, get a chat window and a local OpenAI-compatible server, and start collecting sessions. The recent releases added a session export feature that dumps conversations as JSONL, which is the raw material for a feedback dataset.
Where it falls short: No built-in fine-tuning. LM Studio is inference-first; the training happens elsewhere.
Pricing:
- Free: Everything.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: lmstudio.ai
Bottom line: The default for capturing and inspecting model output on the way to a tuning run.
3. Ollama – best for serving the fine-tuned model afterwards
Ollama is the boring, reliable model server. Once you have fine-tuned a LoRA, wrap it as an Ollama model file and every client that talks to a local OpenAI endpoint gets your improved version instantly. The recent releases added first-class LoRA loading, so you can swap adapters without re-downloading base weights.
Where it falls short: No UI. It is a daemon that expects other tools to sit on top.
Pricing:
- Free: Everything.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: ollama.com — Source (GitHub)
Bottom line: The right pick for hosting the tuned model on your machine.
4. Axolotl – best for reproducible LoRA runs
Axolotl is a YAML-driven wrapper around Hugging Face’s training stack. Point it at a dataset, pick a base model, set your LoRA rank and learning rate, and go. Every run is reproducible from the config file, which matters when the whole point of self-improvement is measuring improvement across iterations.
Where it falls short: Linux-native. Runs on Windows via WSL and on macOS via container but with rough edges.
Pricing:
- Free: Everything.
- Paid: None.
Platforms: Linux primarily; WSL for Windows.
Download: github.com/axolotl-ai-cloud/axolotl
Bottom line: The right pick if you want experiments to be repeatable a month later.
5. Unsloth – best for fast single-GPU LoRA
Unsloth is a fine-tuning library that squeezes 2-5x speed on a single consumer GPU compared with stock Hugging Face Trainer. VRAM use is roughly halved too. On a 4090 that means a 7B LoRA run in an hour instead of four; on a 3060 it means the run finishes at all.
Where it falls short: Some architectures are behind the base transformers release. Support for the newest models comes in a version or two later.
Pricing:
- Free: Everything on a single GPU.
- Paid: Unsloth Pro for multi-GPU and enterprise support.
Platforms: Windows, Linux.
Download: unsloth.ai — Source (GitHub)
Bottom line: The right pick when a single desktop GPU is all you have.
6. Text Generation WebUI – best for manual labelling and evaluation
Text Generation WebUI (Oobabooga) is the workhorse UI for local inference, and it doubles as a labelling and evaluation environment. Load the base and the tuned models side by side, run the same prompt against both, and mark which answer is better. Every rating goes into the same conversation log for the next round.
Where it falls short: The default UI is dated. Setup can be finicky on Windows without WSL.
Pricing:
- Free: Everything.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: github.com/oobabooga/text-generation-webui
Bottom line: The evaluation harness for people who like a UI.
7. DSPy – best when the failure is a prompt, not the weights
DSPy treats prompts as parameters. Give it a small labelled dataset of failures, define a metric, and it optimises the prompt (and few-shot examples) to fit. For a lot of failure modes this is faster and cheaper than a fine-tune. The optimised prompt can then feed back into your production agent.
Where it falls short: Learning curve. The abstractions (Signatures, Modules, Optimizers) are powerful but non-obvious on first read.
Pricing:
- Free: Everything.
- Paid: None.
Platforms: Windows, macOS, Linux.
Download: dspy.ai — Source (GitHub)
Bottom line: Try DSPy before a fine-tune. Cheap prompts fix cheap failures.
How to pick the right one
If you are just getting started and want a single stack, LM Studio for inference plus LangChain for capture plus Unsloth for the LoRA training run is the cheapest path to a working loop.
If your failures are mostly reasoning gaps (the model rambled, missed a step), start with DSPy. Prompt optimisation often closes the gap for cents rather than GPU hours.
If your failures are domain-specific (the model does not know your codebase or product), that is a fine-tune. Use Axolotl or Unsloth on top of the LM Studio and Ollama duo.
If you are on a workstation with two or more GPUs, Axolotl scales further than Unsloth’s free tier.
Use Text Generation WebUI as your evaluation UI regardless of what you train with. Manual A/B tests are the fastest way to know a tune actually improved things.
FAQ
Can I self-improve a local LLM without training on my own data? Only up to a point. Prompt optimisation with DSPy gets you further than most people expect, but domain-specific failures need domain-specific data.
How much data do I need for a useful LoRA fine-tune? For a targeted fix, a few hundred labelled failure examples is enough. Instruction-tuning a general model needs tens of thousands.
Which is faster on a 3060 or 4060: Unsloth or Axolotl? Unsloth, by a wide margin on a single GPU. Axolotl is better once you have multiple GPUs or want reproducible YAML configs.
Do I lose the base model’s abilities when I LoRA-tune? Not usually. LoRA freezes the base weights and adds a small adapter. You can unload the adapter and get the base model back.
Can I run all of this on an Apple Silicon Mac? Inference (LM Studio, Ollama, Text Generation WebUI) yes. Training runs slower on Metal than on CUDA; some architectures still need a Linux GPU box for practical training times.
