Losing a project’s README, ADRs, or API notes is the kind of accident that ruins a Monday. If the codebase is intact, a local LLM can read it back to you and rebuild the docs you lost, page by page, without sending a single line to a cloud provider. We tested seven local-LLM apps that can do this on a modern laptop with 16 to 32 GB of RAM. All of them can walk your source tree, chunk it, embed it, and turn the result into README-quality prose. What they differ on is how much setup they take, how well they cite the source files, and how easily you can hand the output back to Git.
What to look for in a local-doc-reconstruction tool
- Repo-aware chunking. Reads whole folders, respects
.gitignore, and keeps file paths in context. - Citation of source files. Each generated paragraph should link back to the file it came from so you can verify.
- Local embeddings. Anything that ships your code to a hosted embedding endpoint defeats the point.
- Streaming output. README reconstruction can be long. Waiting five minutes for a first token is painful.
- Model choice. A tool that runs only one model boxes you in when a bigger model would give you better prose.
- Diff-friendly output. Markdown that Git can review sensibly, not one huge paragraph.
Quick comparison
| App | Best for | Free plan | Platforms | Rating |
|---|---|---|---|---|
| Ollama | Model backbone for everything else | Free (open source) | Windows, macOS, Linux | GitHub 100k+ stars |
| LM Studio | GUI-first users | Free personal | Windows, macOS, Linux | 4.7 on Product Hunt |
| Continue | VS Code and JetBrains integration | Free (open source) | Windows, macOS, Linux | 4.6 on VS Code Marketplace |
| Aider | Terminal-first, Git-native | Free (open source) | Windows, macOS, Linux | GitHub 20k+ stars |
| GPT4All | One-click chat with local docs | Free (open source) | Windows, macOS, Linux | 4.4 stars average |
| Jan | Privacy-first ChatGPT replacement | Free (open source) | Windows, macOS, Linux | 4.5 stars average |
| Cline | Autonomous VS Code agent | Free (open source) | Windows, macOS, Linux | 4.7 on VS Code Marketplace |
| LocalAI | Self-hosted OpenAI-compatible API | Free (open source) | Windows, macOS, Linux | GitHub 25k+ stars |
The 8 best local LLM apps for reconstructing lost documentation
1. Ollama, the backbone almost every other tool uses
Ollama is the runtime that serves quantised models on your machine over a simple HTTP API. On its own it is a chat CLI. Paired with a doc-reconstruction workflow (feed it your files, ask for a README rebuild), it becomes the engine every other tool on this list can talk to. Pull qwen2.5-coder:32b or llama3.1:70b if you have the VRAM, or qwen2.5-coder:7b if you do not.
Where it falls short: No GUI. No repo-walking out of the box, you script it or pair it with something else.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: Ollama.com GitHub
Bottom line: Install Ollama first even if you plan to use one of the tools below. Every good local workflow eventually leans on it.
2. LM Studio, best GUI for people who want a big chat window
LM Studio is the friendliest desktop client for running local LLMs. Search the model catalogue, download a GGUF file, and chat. The 2025 releases added a workspace mode where you can drag a folder in and ask the model questions about its contents. Doc reconstruction is a “select folder, prompt with ‘rebuild the README’” flow that anyone can follow.
Where it falls short: Not open source (free for personal use). The workspace mode still chunks large repos coarsely, so very big monorepos need pre-processing.
Pricing:
- Free: Free for personal use, paid workplace license required for commercial deployment
- Paid: Workplace license (contact for pricing)
Platforms: Windows, macOS, Linux
Download: LMStudio.ai
Bottom line: Best if you want a click-through experience and are working on your own project. Skip for commercial deployment unless you buy the license.
3. Continue, the local model that lives in your IDE
Continue is a VS Code and JetBrains extension that connects to Ollama, LM Studio, or a local llama.cpp server. Its “codebase context” feature indexes your repo locally and lets you say “regenerate the README for this project” or “rebuild the architecture notes for the src/adapters folder.” Output streams into a side panel and can be dropped into files as you review.
Where it falls short: Setup takes a config file. First-time users sometimes miss the model-provider settings and end up talking to the wrong backend.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: Continue.dev VS Code Marketplace GitHub
Bottom line: Pick Continue if you want doc rebuilding to happen inside your IDE alongside your normal editing. Skip if you dislike IDE-based agents.
4. Aider, terminal-first and Git-native
Aider is a terminal AI coding assistant that treats every change as a Git commit. Point it at Ollama or an OpenAI-compatible endpoint, run aider docs/README.md, and ask for a rebuild. It reads the files you add to the session, produces a diff, and commits the result with a message you can edit. If you like reviewing changes in git diff rather than a chat window, this is your tool.
Where it falls short: Steep first hour. Terminal-only, so if you dislike CLI, this is not for you.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: Aider.chat GitHub
Bottom line: Pick Aider if the diff and the commit message are what you want out of a doc rebuild. Skip if you want a chat UI.
5. GPT4All, the “install one app and go” option
GPT4All is the closest thing to a one-click local LLM chat client. Install it, pick a model from the built-in library, drag a folder into the “LocalDocs” panel, and it indexes the folder locally and lets you chat about the contents. For pure “give me back my README” tasks on small to medium repos, it works out of the box on almost any machine with 16 GB of RAM.
Where it falls short: LocalDocs indexing gets slow past a few thousand files. Model choice is smaller than Ollama’s catalogue.
Pricing:
- Free: Fully free and open source (business licence for commercial use)
- Paid: None for personal use
Platforms: Windows, macOS, Linux
Download: GPT4All.io GitHub
Bottom line: Best “install and forget” option. Skip once your repo passes a few thousand files.
6. Jan, privacy-first ChatGPT-style workspace
Jan is an open-source ChatGPT-style app that runs entirely locally by default. It bundles a model runner, a chat UI, and file attachment support that will read your code files and answer questions. Doc reconstruction feels like chatting with a friend who has actually read the whole repo.
Where it falls short: File indexing is per-conversation rather than a persistent workspace, so large rebuilds sometimes need to be split.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Bottom line: Pick Jan if you want a private ChatGPT that also reads your code. Skip if you want persistent repo memory across sessions.
7. Cline, autonomous rebuild agent in VS Code
Cline (formerly Claude Dev) is a VS Code extension that runs in agent mode: give it a goal, and it walks the repo, opens files, runs commands, and produces output. Pointed at a local Ollama endpoint, it can do “read every file in this repo and rebuild the architecture doc from scratch” as a single task while you go get coffee.
Where it falls short: Autonomous by design, so watch what it edits. The best local models still lag hosted models on multi-step reasoning.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: Cline.bot VS Code Marketplace GitHub
Bottom line: Pick Cline if you want to hand off the whole rebuild as a task rather than steer it turn by turn. Skip if you want tight control.
8. LocalAI, self-hosted OpenAI-compatible API
LocalAI exposes an OpenAI-compatible API over your local models, so any tool that already talks to OpenAI (VS Code extensions, doc generators like Mintlify’s CLI, custom scripts) can talk to your local machine instead. Doc reconstruction becomes “point your existing tool at http://localhost:8080 and generate against your repo without leaking a line.”
Where it falls short: Setup is heavier than Ollama, since it supports many more backends. Model management via Docker is fine for developers but not casual users.
Pricing:
- Free: Fully free and open source
- Paid: None
Platforms: Windows, macOS, Linux
Download: LocalAI.io GitHub
Bottom line: Pick LocalAI if you already have a doc-generation script pointed at OpenAI and just want to redirect it locally. Skip if Ollama already covers what you need.
How to pick
- If you want the smallest install: GPT4All.
- If you live in VS Code: Continue for interactive rebuild, Cline for hands-off.
- If you live in the terminal: Aider, with Ollama as the backend.
- If you have a bunch of scripts pointed at OpenAI already: LocalAI.
- If you want a chat window with attached files: LM Studio for polish, Jan for full open source.
- If you have serious hardware (24 GB VRAM or an M4 Max): run
qwen2.5-coder:32bthrough Ollama and drive any of the above.
FAQ
What model should I use for doc reconstruction?
qwen2.5-coder:32b is the sweet spot on 24 GB of VRAM. On 16 GB, use qwen2.5-coder:7b. On Apple Silicon with 32 GB unified memory, qwen2.5-coder:32b runs comfortably.
Does this actually keep my code off the internet?
Yes, when configured correctly. Every tool on this list can be run against a local model. If you configure them to use a hosted API key, they no longer are local.
How long does it take to rebuild a README for a medium repo?
Roughly 3 to 10 minutes on a modern laptop with a mid-size model. Very large repos need to be split by module.
Can a local LLM rebuild real API documentation?
Yes, for structural docs (endpoints, params, response shapes) if the code is typed. Prose descriptions of intent still benefit from a human editing pass.
Which tool should I try first?
GPT4All if you want zero setup. LM Studio if you want a nicer GUI. Continue if you already use VS Code.
