LM Studio and Ollama made running local LLMs a one-click affair. Fine-tuning them stayed a datacenter job for years, out of reach on the same hardware. That is finally changing. Unsloth’s new desktop app is the visible sign, but a small cluster of open frameworks now trains a real LoRA adapter on a single 12 GB or 24 GB consumer GPU in the space of an afternoon.
We tested seven tools across an RTX 4070, an RTX 4090, and a base M2 Pro Mac. The picks below are the ones that finished a Llama 3.1 8B or Mistral 7B LoRA run without running out of memory, without needing a cluster, and without asking you to write a training loop in PyTorch first. The list mixes GUI-first tools with CLI-first frameworks so you can pick the ergonomic level you want.
What to look for in a consumer-GPU fine-tuner
- LoRA and QLoRA support, so a 7B model trains on 12 GB and a 13B on 24 GB.
- 4-bit and 8-bit quantized loading, which is the reason any of this fits.
- Flash Attention or an equivalent kernel so training speed does not fall off a cliff.
- Support for the model families you actually care about: Llama, Mistral, Qwen, Gemma, Phi.
- Dataset formats that are easy to prepare, ideally JSONL with instruction/output fields.
- Checkpointing so a crash at hour four does not cost you the whole run.
Quick comparison
| App | Best for | Platforms | Free plan | Starting price/mo | License |
|---|---|---|---|---|---|
| Unsloth | Fastest LoRA on 12–24 GB | Windows, Linux | Full app | Free (Pro tier) | Apache 2.0 |
| Axolotl | Config-file power user | Linux | Full app | Free | Apache 2.0 |
| LLaMA-Factory | GUI-driven, wide model support | Windows, Linux | Full app | Free | Apache 2.0 |
| Text Generation WebUI | Attach a LoRA and test in-place | Windows, macOS, Linux | Full app | Free | AGPLv3 |
| PEFT | Library for your own training loop | Windows, macOS, Linux | Full app | Free | Apache 2.0 |
| H2O LLM Studio | Corporate UI with experiment tracking | Windows, Linux | Free (open source) | Enterprise support | Apache 2.0 |
| MLX-LM | Native fine-tuning on Apple Silicon | macOS | Full app | Free | MIT |
The apps
1. Unsloth, best fastest LoRA on 12–24 GB
Unsloth rewrites the hot paths of Llama, Mistral, Qwen, Gemma, and Phi fine-tuning to be roughly two to five times faster and about half the memory of stock Transformers. A 7B LoRA fits on a single 12 GB GPU and finishes on a modest dataset in an afternoon. The new desktop app removes the last CLI step for the common recipes.
Where it falls short: coverage skews toward the most popular model families; obscure architectures need the upstream Transformers path.
Pricing:
- Free: Full framework and desktop app.
- Paid: Pro tier for enterprise support and faster multi-GPU kernels.
Platforms: Windows (via WSL2), Linux.
Download: unsloth.ai · GitHub
Bottom line: The default choice in 2026 for LoRA training on a single consumer card.
2. Axolotl, best config-file power user
Axolotl is a config-driven training framework built around Hugging Face. A single YAML file describes the model, dataset, LoRA rank, and training args, and Axolotl handles the rest. It supports fewer speed optimizations than Unsloth but wins on flexibility across less-common models and dataset formats.
Where it falls short: no GUI; the config has a learning curve; docs assume you have trained something before.
Pricing:
- Free: Full app.
- Paid: Not applicable.
Platforms: Linux (Docker on macOS/Windows).
Download: GitHub (axolotl-ai-cloud/axolotl)
Bottom line: The right pick when Unsloth does not have your model and you still want a sane pipeline.
3. LLaMA-Factory, best GUI-driven trainer
LLaMA-Factory ships a Gradio-based GUI on top of a broad training toolkit. Choose a model, upload a dataset, set LoRA rank and learning rate, and hit start. It covers 100+ model families out of the box and its charts make hyperparameter tweaking approachable.
Where it falls short: the GUI hides some of the harder choices; on VRAM edge cases it needs the CLI anyway.
Pricing:
- Free: Full app.
- Paid: Not applicable.
Platforms: Windows, Linux.
Download: GitHub (hiyouga/LLaMA-Factory)
Bottom line: The best GUI-first entry point for anyone new to LoRA training.
4. Text Generation WebUI, best for attaching and testing a LoRA
Text Generation WebUI (“oobabooga”) is the local-LLM chat UI most enthusiasts already run. Its Training tab does LoRA training in place, and once a run finishes you can enable the adapter and chat with the model to sanity-check the result immediately.
Where it falls short: not the fastest trainer on this list; the training tab is a good “connect the dots” tool, not a production pipeline.
Pricing:
- Free: Full app.
- Paid: Not applicable.
Platforms: Windows, macOS, Linux.
Download: GitHub (oobabooga/text-generation-webui)
Bottom line: The lowest-friction tool to see if fine-tuning your own data changes anything you care about.
5. PEFT, best library for your own training loop
PEFT is Hugging Face’s Parameter-Efficient Fine-Tuning library. It is a set of Python building blocks, LoRA, IA3, prefix tuning, that you drop into a Trainer. If you want to control the loop, this is the level you work at.
Where it falls short: no UI; every experiment is a script; you are responsible for the ergonomics.
Pricing:
- Free: Full app.
- Paid: Not applicable.
Platforms: Windows, macOS, Linux.
Download: GitHub (huggingface/peft)
Bottom line: The right level of abstraction for a researcher or a serious hobbyist who wants full control.
6. H2O LLM Studio, best corporate-flavored trainer
H2O LLM Studio wraps the same open frameworks in a clean React UI, adds experiment tracking, and integrates cleanly with H2O’s cloud tools when a run outgrows a single GPU. On a workstation with a 4090 it is comfortable.
Where it falls short: the UI is a heavier install than any other pick here; opinionated defaults can hide the underlying knobs.
Pricing:
- Free: Full open-source app.
- Paid: Enterprise support.
Platforms: Windows, Linux.
Download: GitHub (h2oai/h2o-llmstudio)
Bottom line: The pick when experiments are going to a team, not a single laptop.
7. MLX-LM, best native fine-tuning on Apple Silicon
MLX-LM is the Apple-native LM framework built on MLX. On a Mac Studio or a base M2 Pro with 32 GB unified memory, LoRA fine-tuning on 7B and 13B models is now a supported path, and the framework’s memory model uses the shared GPU/CPU allocation to fit models CUDA never would.
Where it falls short: macOS only; the model catalog is smaller than CUDA’s; less mature than PyTorch-based tools.
Pricing:
- Free: Full app.
- Paid: Not applicable.
Platforms: macOS (Apple Silicon).
Download: GitHub (ml-explore/mlx-lm)
Bottom line: The right pick on a modern Mac, the wrong pick anywhere else.
How to pick the right one
If you have a single 12 GB or 24 GB consumer GPU and want speed, install Unsloth and run one of its notebooks. Nothing else on this list gets to a working adapter faster.
If your model or dataset format is outside Unsloth’s fast path, pick Axolotl for its config-driven flexibility.
If you have never trained anything before, LLaMA-Factory puts a GUI on the same underlying tools and shortens the “which learning rate” debate.
If you already run Text Generation WebUI, use its Training tab. Do not add another tool for a one-off run.
For control over the training loop, PEFT is the right library. Expect to write Python.
If experiments need to be reviewed by other humans, H2O LLM Studio ships that UI in the open-source tier.
MLX-LM is the correct choice on Apple Silicon and only there.
FAQ
Do I need a 24 GB GPU to fine-tune a 7B model?
No. Unsloth and QLoRA fit a 7B LoRA on 8–12 GB. 24 GB gets you 13B comfortably.
Can I fine-tune on a CPU?
Technically yes, in the way you can technically cross an ocean by rowboat. Use a GPU or Apple Silicon.
Will fine-tuning make my local model better at my job?
For instruction following and domain vocabulary, yes. For hard reasoning gains, mostly no; the base model’s ceiling is the ceiling.
How large should my dataset be?
For a task-focused LoRA, 500 to 5,000 high-quality examples beat 100,000 mediocre ones. Curate ruthlessly.
Can I share the LoRA adapter I trained?
Yes; adapters are small. Check the underlying base model’s license first, most Llama and Mistral variants are permissive but not identical.