An XDA piece this week described training a local LLM on its own failures. The author kept the model’s wrong outputs, hand-graded a few dozen, packed them into a small dataset, and ran a lightweight fine-tune. The result was a noticeably better model on the exact problems the base version had been getting wrong. What used to want a rented cluster now runs on a laptop with a 12 GB GPU. These are the seven best apps for local LLM fine-tuning on desktop in 2026, all free and open source, ranked by how quickly a first fine-tune actually runs.
What to look for in a local fine-tuning app
- LoRA and QLoRA support. Full fine-tunes are hardware-bound. Low-rank adapters (LoRA) and 4-bit quantised LoRA (QLoRA) are the only realistic way to fine-tune a 7B or 13B model on consumer GPUs.
- Runs on the hardware you have. NVIDIA CUDA is the default; ROCm on AMD is second-class. Apple Silicon (M-series) has its own path through MLX. A “fine-tune app” is only useful if it runs on your GPU or NPU.
- A dataset format that isn’t scary. JSON Lines with
instruction+outputfields is the modern floor. If the app wants a custom binary format, it will fight you. - Evaluation and checkpointing. Fine-tuning goes wrong. A tool that snapshots every N steps and shows loss curves lets you back off a bad run before you burn the weekend.
- Exports to GGUF or MLX. Once fine-tuned, the model needs to load into an inference stack. GGUF for llama.cpp; MLX for Apple. If the tool locks the weights to its own format, you can’t ship the result.
Quick comparison
| App | Best for | Backend | LoRA/QLoRA | Apple Silicon | Ease |
|---|---|---|---|---|---|
| Unsloth | Fastest single-GPU LoRA | CUDA | Yes | Beta | Highest |
| Axolotl | Configurable production runs | CUDA (+ROCm) | Yes | No | Medium |
| LLaMA-Factory | Web-UI fine-tuning | CUDA (+ROCm) | Yes | Beta | High |
| MLX-LM | Apple Silicon fine-tunes | MLX | Yes | Yes | High |
| Torchtune | Official PyTorch reference | CUDA | Yes | Limited | Medium |
| Hugging Face TRL | RLHF/DPO/PPO on top of Transformers | CUDA (+ROCm) | Yes | No | Medium |
| LM Studio | GUI for inference + light fine-tune workflows | CUDA/Metal | Adapter loading | Yes | Very high |
1. Unsloth, Best for the fastest single-GPU LoRA
Unsloth is the tool that turned “fine-tune a 7B model on your laptop” from a research paper into a Colab notebook and a pip install. On a single RTX 4070 with 12 GB of VRAM, a 7B QLoRA run at 2048 context finishes in around 30 minutes for a few hundred examples. The math trick is a memory-efficient forward-backward pass; the practical benefit is a fine-tune that runs while you make coffee.
Where it falls short: Multi-GPU training is second-class compared to Axolotl or Torchtune. Some niche architectures land later than in the reference stacks.
Pricing: Free.
Platforms: Linux (best), Windows (via WSL2), Apple Silicon (beta).
Download: unsloth.ai / github.com/unslothai/unsloth
Bottom line: The default first fine-tune for anyone with a single consumer GPU.
2. Axolotl, Best for configurable production runs
Axolotl is what teams use when Unsloth’s happy-path notebook isn’t enough. The config-file approach makes runs reproducible: a single YAML file describes the base model, dataset, LoRA rank, learning rate, evaluation cadence, and export target. Multi-GPU with DeepSpeed and FSDP works. The community has published dozens of preset configs for common tasks.
Where it falls short: The learning curve is steeper than Unsloth’s. First run means reading a YAML config and understanding what each field does.
Pricing: Free.
Platforms: Linux, Windows via WSL2.
Download: github.com/axolotl-ai-cloud/axolotl
Bottom line: The tool to graduate to after the first Unsloth fine-tune worked and you want more control.
3. LLaMA-Factory, Best for a web UI fine-tuning workflow
LLaMA-Factory wraps the same math Axolotl and Unsloth do behind a web UI. Load a base model, pick a dataset from a local folder, adjust a slider for LoRA rank, and start the run. Loss curves render in the browser. For someone who doesn’t want to touch a terminal, this is the closest thing to a friendly desktop fine-tuning app.
Where it falls short: The web UI is not a full IDE. Complex multi-stage pipelines still want a real config. Not every optimiser is exposed in the UI.
Pricing: Free.
Platforms: Linux, Windows via WSL2, macOS via Docker.
Download: github.com/hiyouga/LLaMA-Factory
Bottom line: The best pick if a terminal-first workflow puts you off.
4. MLX-LM, Best for Apple Silicon fine-tunes
MLX-LM is Apple’s official language-model stack, built on the MLX array framework. On an M2 Max or M3 Max with 64 GB of unified memory, a 7B LoRA fine-tune runs at speeds competitive with a mid-range NVIDIA GPU. On an M2 Pro with 16 GB it works for smaller models. MLX-LM exports directly to MLX format for Ollama and Apple’s own inference runtimes.
Where it falls short: Not every base model is available in MLX format; the conversion step from Hugging Face weights adds friction. Ecosystem is smaller than the CUDA stack.
Pricing: Free.
Platforms: Apple Silicon macOS only.
Download: github.com/ml-explore/mlx-lm
Bottom line: The right tool on a Mac. Nothing else uses Apple Silicon this well.
5. Torchtune, Best for the official PyTorch reference
Torchtune is PyTorch’s own fine-tuning library, run by Meta’s team. Recipes are readable Python, not a config DSL, which suits engineers who want to see and modify the training loop. Support for full fine-tunes, LoRA, and DPO. Multi-GPU works out of the box. The docs are engineering-grade.
Where it falls short: Less “batteries included” than Unsloth or Axolotl. The reference-implementation posture means fewer shortcuts.
Pricing: Free.
Platforms: Linux, Windows via WSL2, macOS with limited features.
Download: github.com/pytorch/torchtune
Bottom line: Pick Torchtune if you already write PyTorch and want to see the training loop.
6. Hugging Face TRL, Best for RLHF, DPO, and reward modelling
TRL (Transformer Reinforcement Learning) is the toolkit for the fine-tuning steps beyond a plain supervised fine-tune: DPO (direct preference optimisation), PPO (proximal policy optimisation), reward-model training. On the XDA “train on failures” workflow, DPO is the actual technique that turned graded failures into a better model. TRL is what runs it.
Where it falls short: More math to understand. RLHF/DPO can go wrong in ways SFT cannot; expect a few failed runs before it clicks.
Pricing: Free.
Platforms: Linux, Windows via WSL2.
Download: github.com/huggingface/trl
Bottom line: The tool for the second stage of a real “learn from your mistakes” fine-tune pipeline.
7. LM Studio, Best for an all-in-one desktop GUI
LM Studio is not primarily a fine-tuning app. It’s a desktop GUI for downloading, running, and chatting with local models on Windows, macOS, and Linux. Its role in this list is the polish it provides around a fine-tune workflow: load a base model, load a LoRA adapter produced by Unsloth or Axolotl, chat with the fine-tuned model, iterate. The 2026 releases added light fine-tune convenience wrappers that call out to Unsloth under the hood.
Where it falls short: The training side is thin. LM Studio is a chat and inference frontend; use it for the inference half of the loop, not the training half.
Pricing: Free.
Platforms: Windows, macOS, Linux.
Download: lmstudio.ai
Bottom line: Pick LM Studio as the inference side of the fine-tune loop, not the training side.
How to pick the right one
- Never fine-tuned a model before, have a single NVIDIA GPU: Unsloth. Follow the QLoRA notebook first.
- On an Apple Silicon Mac: MLX-LM. Nothing else uses the hardware this well.
- Want a web UI and no terminal: LLaMA-Factory.
- Building a reproducible pipeline for a team: Axolotl with YAML configs in Git.
- Doing the XDA-style “train on failures” workflow: Unsloth for the initial SFT, then TRL for DPO on the graded failures.
- Want to see the training loop in plain PyTorch: Torchtune.
Pair whichever training tool you pick with LM Studio for testing the resulting adapter locally before shipping it into an inference stack.
FAQ
What is the minimum GPU for fine-tuning a 7B model?
QLoRA at 2048 context fits in 8 GB of VRAM for the smallest 7B variants and 12 GB comfortably. 8 GB works for shorter contexts and lower ranks; 24 GB opens 13B QLoRA.
Can we fine-tune without any GPU at all?
Technically yes, on CPU, but it’s slow to the point of impractical. Cloud rental for a few hours is often cheaper than the electricity to run a CPU fine-tune to completion.
What data format do these tools accept?
JSON Lines with instruction and output fields (Alpaca-style) is the universal floor. Most tools also accept ShareGPT format and OpenAI’s chat message format.
Do the fine-tuned models run in Ollama, LM Studio, or llama.cpp?
Yes, after exporting to GGUF (for llama.cpp and Ollama) or MLX (for Apple’s stack). Every tool above exports to at least one of those formats.
Is fine-tuning legal for closed-model weights?
Only for models whose licence permits it. Llama 3 and 4, Mistral models, Qwen, Gemma, and every open-weight model in this list allow fine-tuning under their licences. Closed models (GPT-4 class) do not have downloadable weights, so no local fine-tune is possible.