The strongest self-hosted AI stack right now pairs Ollama, for running open models on hardware you own, with Open WebUI for a ChatGPT-style interface, then adds n8n once you want those models to take actions instead of just answering questions. All three run in Docker on a machine you probably already have running Jellyfin or Home Assistant, with no subscription, no per-token bill, and nothing leaving your network.
🇫🇷 Version française : retrouvez cet article en français dans Meilleurs outils d’IA locale pour votre homelab en 2026
What this list actually covers: every tool below is open source, or at least free to self-host, and runs entirely on hardware you control. Nothing here is a hosted API wearing a “private AI” label. If a prompt has to leave your network to reach someone else’s server, it didn’t make the cut.
What does “self-hosting AI” actually mean?
Self-hosting AI means running large language models, and the tools built around them, on your own laptop, homelab server, or a box you fully control, instead of sending every prompt to OpenAI, Anthropic, or Google. The trade-off is an honest one: you give up some raw capability and a bit of convenience in exchange for full control over your data, zero per-token billing, and a system that keeps working even when your internet doesn’t.
Why self-hosted AI is suddenly everywhere
Search interest in self-hosted AI has climbed several-fold over the past year, even though the absolute numbers are still a fraction of how often people search for ChatGPT itself. Three things are driving it at once: open models like Llama, Mistral, and Qwen have gotten good enough to be genuinely useful instead of just a novelty, older GPUs have gotten cheaper as newer ones launched, and a steady stream of AI privacy stories has made “my prompts never leave my own hardware” sound less paranoid and more sensible. None of that is unique to professional developers anymore. It’s the same audience already running Jellyfin instead of paying for another streaming subscription, applying the same instinct to chatbots.
What hardware do you need?
You don’t need a server rack to start. Roughly 8GB of RAM and no GPU at all is enough to run 1B-3B parameter models like Llama 3.2 1B on CPU alone, just slower than you’re used to from ChatGPT. For useful 7B-8B models like Llama 3.1 8B at a comfortable speed, aim for 16GB of RAM plus a GPU with 8GB or more of VRAM. Heavier use, 13B-30B models and coding assistants, wants 32GB of RAM and 16-24GB of VRAM. As a rough rule of thumb, budget about 0.5GB of VRAM per billion parameters at 4-bit quantization.
None of this requires new hardware if you’re already running a homelab. A mini PC or a repurposed desktop that’s already handling Jellyfin or a NAS can run a small model in the background without breaking a sweat. A GPU only becomes worth buying once you want 7B-or-larger models to feel instant instead of merely usable.
The best self-hosted AI tools at a glance
Six tools cover almost everything most homelabs actually want from self-hosted AI, from running a model to putting it to work.
| Category | License | GPU needed? | |
|---|---|---|---|
| Ollama | LLM runtime | MIT | Optional, much faster with one |
| Open WebUI | Chat interface | Open WebUI License (BSD-3-based, branding clause since v0.6.6) | No, just needs a backend like Ollama |
| n8n + starter kit | AI workflows & agents | Sustainable Use License (n8n) / Apache-2.0 (starter kit) | No |
| LocalAI | OpenAI-compatible runtime | MIT | No |
| Immich | AI photo search & face recognition | AGPL-3.0 | Optional, speeds up indexing |
| Home Assistant + Ollama | Local voice & home automation | Apache-2.0 | Recommended for fast replies |
Getting started: the fastest path to a working setup
You don’t need to install all six tools at once. Here’s the order that gets you to something useful the fastest:
- Install Ollama on your server or homelab machine, a one-line installer script on Linux and macOS, a standard installer on Windows.
- Pull a model that fits your RAM.
ollama run llama3.2downloads and starts chatting with a small model in one command. - Add Open WebUI in Docker once you want a browser-based interface instead of a terminal window.
- Add n8n, or the full self-hosted AI starter kit, once you want your models to take actions instead of just answering questions.
Ollama: the engine for running models locally
Ollama is the piece almost everything else on this list assumes you already have. It’s a free, MIT-licensed command-line tool that downloads open models and serves them through a local API, so anything else, a chat interface, an automation tool, a custom script, can talk to your models the same way it would talk to OpenAI’s API. Getting a model running is a single command.
ollama run llama3.2That’s genuinely the whole install for a first test. Ollama handles quantization, GPU acceleration, and memory management for you, which is most of why it’s become the default answer to “how do I run an LLM at home” across r/selfhosted and r/homelab alike. The official Ollama site lists every model in its library along with the RAM each one needs, and the project’s source is on GitHub if you want to see exactly what’s running on your machine.
Which model you pull matters more than which tool you use to run it. Llama, Mistral, Qwen, Gemma, and DeepSeek all publish versions small enough for modest hardware, and Ollama’s library notes which ones fit an 8GB or 16GB machine before you download anything. Start with whichever recent general-purpose model fits your RAM; switching to a different one later never requires reinstalling Ollama itself.
Open WebUI: the ChatGPT-style interface
Open WebUI is what turns Ollama’s API into something that looks and feels like ChatGPT: a browser-based chat interface with conversation history, multiple models, file uploads, and image generation support if you’ve got a diffusion model running alongside it. It’s the interface most self-hosted AI tutorials build their whole setup around, for good reason, it’s the fastest way to hand a local model to someone who has no interest in living in a terminal.
Tip: Open WebUI’s own Docker image can bundle Ollama in as a sidecar container, so you can skip a separate install step and go straight to the interface if you’d rather not run two setup processes.
One thing worth knowing if license purity matters to you: Open WebUI’s code was BSD-3-Clause through version 0.6.5, but releases from v0.6.6 onward ship under a modified license that requires preserving Open WebUI’s branding unless you meet specific exemption criteria the project documents on its license page. It’s still free to self-host, use, and modify; it’s just no longer an OSI-approved open source license in the strictest sense, which is exactly the kind of detail the self-hosted crowd tends to ask about before installing anything. The project’s source remains publicly visible on GitHub.
What you’ll have: a ChatGPT-style interface running at your server’s local address, talking to whatever models you’ve pulled with Ollama, with no account, no subscription, and every conversation stored only on your own disk.
n8n: turning your models into agents, not just chatbots
Once a chatbot isn’t enough, n8n is the tool most of this niche reaches for. It’s a low-code automation platform with over 400 integrations, and n8n’s own self-hosted AI starter kit bundles it with Ollama, the Qdrant vector database, and PostgreSQL in a single Docker Compose file, everything needed to build AI agents that summarize documents, watch a folder, or post to Slack, without any of it touching a third-party API. The starter kit repo has picked up close to 15,000 GitHub stars since launch, for a project that’s essentially a curated docker-compose.yml file, a decent sign of how much demand there is for exactly this combination.
docker compose --profile cpu upThat one command, or a GPU-specific profile if you’ve got an Nvidia or AMD card, brings up the entire stack. n8n itself runs under its own Sustainable Use License, source-available and free for personal or internal use; the restriction only kicks in if you resell n8n itself as a hosted service to other people. The starter kit’s own glue code, separate from n8n’s core, is plain Apache-2.0.
LocalAI: the OpenAI-compatible alternative to Ollama
If you’ve already got scripts or tools built against OpenAI’s API and don’t want to rewrite them, LocalAI is worth a look before Ollama. It’s MIT-licensed, implements the same /v1/chat/completions and /v1/embeddings endpoints OpenAI’s API uses, and runs LLMs, image generation, and transcription without requiring a GPU. It won’t feel as instantly familiar as Ollama’s one-line model downloads, but it’s the more direct drop-in replacement if “point my existing OpenAI integration at localhost” is the actual goal. The project’s source and full backend list are on GitHub.
If it’s private AI-powered web search you’re after rather than a chatbot, Perplexica and SearXNG do the same job for search engines, both open source, both self-hosted, both built to answer questions without sending your queries to Google.
Beyond chatbots: AI already hiding in your other self-hosted apps
Two apps a lot of homelabbers already run have quietly picked up real local AI features, worth knowing about even if “self-hosted AI” isn’t why you originally installed them.
Immich, the self-hosted Google Photos alternative, runs Smart Search and Facial Recognition through a dedicated immich-machine-learning container, kept separate from the main server so you can offload it to a more powerful machine if yours is underpowered. Smart Search uses CLIP to let you type something like “birthday cake with candles” into the search bar and get ranked, relevant results back, no manual tagging involved. Facial Recognition works the same way, using face embeddings instead of scene descriptions.
Home Assistant can use Ollama directly as a local conversation agent: point its official Ollama integration at a running Ollama server, wire it into an Assist voice pipeline alongside Whisper and Piper over the Wyoming protocol, and you’ve got voice control that never touches a cloud assistant. The catch is model choice: pick something small and fast with good function-calling rather than the biggest model you can run. A GPU with 8GB of VRAM typically keeps replies under two seconds, which is the difference between “usable” and getting up to flip the switch yourself anyway.
Don’t expose your AI stack to the internet without a plan
Ollama binds to your local machine only (127.0.0.1) by default and ships no built-in authentication. That’s fine until someone sets OLLAMA_HOST to 0.0.0.0 to reach it from another device on the network, then forwards the port to reach it from outside the house too. At that point, anyone who finds it can run prompts on your GPU, pull your models, or push new ones. Ollama’s own documentation is explicit that authentication has to be added externally, it isn’t built in.
Treat your AI stack exactly like we’d recommend for a Jellyfin server: put it behind a reverse proxy with real authentication in front, or better, skip exposing it publicly at all and reach it over a mesh VPN like Tailscale or WireGuard instead.
How much does self-hosted AI really cost?
Search around and you’ll find real numbers like $15,000 to $50,000 a month for self-hosted AI. Those figures describe a company renting a rack of top-tier GPUs to match a frontier model’s performance, not a homelab. Running Ollama and Open WebUI on hardware you already own for Jellyfin or Nextcloud costs you electricity and nothing else. The only real spend most homelabbers make is an optional GPU upgrade, a one-time cost, and only if 7B-or-larger models feeling instant matters more to you than saving the money. Either way, there’s no per-token bill waiting at the end of the month.
Self-hosted AI vs ChatGPT: which one should you use?
It doesn’t have to be a competition you pick a side in. Hosted models like ChatGPT, Claude, and Gemini still win on raw capability and zero setup time, they’re simply better at genuinely hard reasoning and coding tasks right now. Self-hosted AI wins on privacy, a fixed one-time hardware cost instead of a recurring subscription, and working with no internet connection at all. Plenty of the homelab crowd runs both: a local model for everyday drafting, summarizing, and questions they’d rather not send anywhere, and a hosted model kept around for the handful of tasks that need it.
If you’re already running a homelab for Jellyfin or Proxmox, adding an AI stack alongside it is a natural next step, it lives on the same box, using the same Docker Compose habits you’ve already got. Our Self-Hosted AI hub and Homelab hub will round up dedicated setup guides for each tool above as we publish them, and the Docker & Containers hub covers the container basics every one of them depends on.
FAQ
Is self-hosted AI actually free?
The software is. Ollama, LocalAI, Immich, and Home Assistant’s Ollama integration cost nothing, and n8n’s Community Edition is free to self-host for personal or internal use. Your only real cost is the hardware you run it on and the electricity it uses.
What’s the easiest way to start self-hosting AI?
Install Ollama, run ollama run llama3.2, and add Open WebUI on top once you want a browser interface instead of a terminal. That’s a working local AI setup in well under fifteen minutes on almost any machine from the last decade.
Do I need a GPU to self-host AI models?
No, not for small models. A CPU-only machine with 8GB of RAM handles 1B-3B parameter models fine, just slower than a GPU would. A GPU becomes worth buying once you want 7B-or-larger models to feel instant rather than merely usable.
Is self-hosted AI as good as ChatGPT or Claude?
For the hardest reasoning and coding tasks, no, frontier hosted models still lead. For drafting, summarizing, home automation, and everyday questions, a well-chosen local model running on modest hardware is close enough that most people stop noticing the gap.
Is it legal to self-host AI models?
Yes. Ollama, Open WebUI, n8n, and the open models they run are all legitimate open-source or source-available software, no different legally from installing any other program on hardware you own.
Can Ollama, Open WebUI, and n8n all run on the same machine?
Yes, and that’s the normal setup. All three are lightweight enough to run together in Docker on a single mini PC or homelab server; n8n’s own self-hosted AI starter kit is built around exactly that combination plus a vector database.
Leave a Reply