Self-Hosted LLM in 2026: The Complete Ollama + Open WebUI Setup Guide

A self-hosted LLM setup pairs Ollama, which downloads and runs open models like Llama or Gemma directly on your own hardware, with Open WebUI, which wraps that model in a ChatGPT-style browser interface with accounts and chat history. This guide walks through installing both from scratch, pulling your first model, and locking the stack down: a working private AI assistant in well under an hour.

What you need before you start

How much hardware does a self-hosted LLM actually need?

Model size, not license or brand, is what decides whether a model runs at a usable speed on your hardware. Here’s the practical breakdown, matched to an actual command you can run at each tier.

Tier	Try this first	RAM (CPU only)	GPU VRAM	Good for
Small (1B-3B params)	ollama run llama3.2:3b	8 GB	None needed	Quick tests, older hardware, a spare Raspberry Pi-class box
Mid (7B-8B params)	ollama run llama3.1:8b	16 GB	8 GB+	The daily-driver sweet spot for most homelabs
Large (13B-14B params)	ollama run qwen2.5:14b	32 GB	16-24 GB	Coding help, longer documents, more nuanced answers
Frontier-class (70B+ params)	ollama run llama3.3:70b	64 GB+	24-48 GB+ or multiple GPUs	Near-cloud quality, a dedicated rig

If you’re not sure where you land, start small. Every model in Ollama’s library lists its own RAM requirement before you download it, and switching to a different one later is a one-line command, not a reinstall.

Step 1: Install Ollama

Ollama installs the same way whether you’re on a laptop or a homelab server: one command, no dependencies to chase down first.

macOS and Linux

curl -fsSL https://ollama.com/install.sh | sh

macOS users who’d rather avoid the terminal entirely can grab the .dmg installer straight from Ollama’s download page instead.

Windows

irm https://ollama.com/install.ps1 | iex

Run that from PowerShell, or download the OllamaSetup.exe installer if you’d rather use a normal Windows install wizard.

Docker

Prefer to keep Ollama contained? The official image handles CPU-only and GPU setups with almost the same command.

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Once installed, running ollama on its own opens an interactive menu, press enter to launch a model, or just skip ahead to the next step.

Step 2: Pull and run your first model

This is the part that used to require a Python environment, a CUDA install, and a free afternoon. Ollama collapses all of it into one line.

ollama run llama3.2

The first run downloads the model, several gigabytes depending on size, then drops you into a chat prompt the moment it’s ready. Every run after that starts in a couple of seconds, since the model is already sitting on disk. Ollama’s own default pick for a quick first test is Gemma 3 (ollama run gemma3), and the full library at ollama.com/library lists dozens more: Llama, Qwen, Mistral, DeepSeek, and OpenAI’s open-weight gpt-oss among them, each page noting the RAM it needs before you pull anything.

If you’d rather script against it than type into a terminal, Ollama also exposes a REST API on port 11434 (curl http://localhost:11434/api/chat -d '{...}'), plus official Python and JavaScript libraries on GitHub, so anything you build later talks to the same running models.

Step 3: Install Open WebUI

A terminal window is fine for testing, but it’s not what you hand to the rest of your household, or use from your phone. Open WebUI turns your Ollama models into something that looks and feels like ChatGPT.

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Visit http://localhost:3000 once it’s running, and if Ollama is already on the same machine, Open WebUI finds it automatically. If you’d rather manage both containers together, or you’re setting this up on a fresh server, a docker-compose.yml is the cleaner long-term option:

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open-webui:/app/backend/data
    restart: unless-stopped

volumes:
  ollama:
  open-webui:

docker compose up -d

Step 4: Connect the two and create your admin account

If you used the combined docker-compose.yml above, Open WebUI already points at Ollama through the OLLAMA_BASE_URL variable. Running them on separate machines instead? Set that same variable to your Ollama server’s address, or add it from the interface under Settings > Admin Settings > Connections once you’ve logged in.

The first account you create in Open WebUI automatically becomes the Administrator, with full control over users and settings. Every account after that starts in a Pending state until an admin approves it, real access control that Ollama’s bare API doesn’t have on its own. That distinction is worth sitting with for a second: Ollama’s API has no built-in login, anyone who can reach port 11434 can use it, while Open WebUI’s accounts are what actually keep a shared household or team setup from turning into a free-for-all.

Configuring Open WebUI for daily use

A few settings turn a working install into one you’ll actually keep using:

1. Pull additional models straight from the interface: Admin Settings > Models > pull a model by name, no terminal required after the first one.
2. Turn on document chat by uploading a PDF or text file into a conversation; Open WebUI handles the retrieval step itself once an embedding model like nomic-embed-text is available.
3. Set a permanent WEBUI_SECRET_KEY environment variable, generated with openssl rand -hex 32, so you’re not logged out every time the container restarts.
4. Invite other household accounts once you trust the setup, then approve each one from the admin panel.

Securing and updating your stack

Both projects update the way you’d expect from anything running in Docker: pull the new image, remove the old container, start it again with the same volume attached so nothing’s lost.

docker pull ghcr.io/open-webui/open-webui:main
docker rm -f open-webui
docker compose up -d

Ollama updates just as easily: re-run the install script on macOS, Linux, or Windows and it replaces itself in place, or docker pull ollama/ollama and recreate the container if that’s how you’re running it. Open WebUI’s Watchtower option automates this if you’d rather not think about it at all.

Troubleshooting common problems

A handful of issues account for most of the “it’s not working” moments with this stack.

• Open WebUI shows no models available. Almost always an OLLAMA_BASE_URL pointing at the wrong address: localhost inside a container refers to the container itself, not your host machine or the other container. Use the service name (ollama) in Docker Compose, or host.docker.internal when Ollama runs directly on the host.
• Responses are unusably slow. Run ollama ps to check whether a model is actually using your GPU or quietly fell back to CPU. If you just installed GPU drivers or the Container Toolkit, a full reboot often fixes detection issues a service restart won’t.
• Connection refused on port 11434. Confirm Ollama is actually running (curl http://localhost:11434/api/tags should return a JSON list) and that nothing else, a firewall rule you forgot about, another service, has claimed the port.
• Models vanish after a container restart. The volume wasn’t mounted, or was mounted under a different name than last time. Named volumes (ollama:/root/.ollama) persist across restarts; anonymous ones don’t.

Is a self-hosted LLM actually worth it?

For everyday drafting, summarizing, and questions you’d rather not send to someone else’s server, yes. A mid-size model on hardware you already own gets close enough to a hosted model that most people stop noticing the gap after the first week. For the hardest coding and reasoning tasks, hosted frontier models still win, and probably will for a while yet, no point pretending otherwise. Plenty of people running this exact stack keep both: Ollama and Open WebUI for daily use, a hosted model kept around for the handful of tasks that actually need it. If you’re still weighing this against everything else self-hosted AI can mean, agents, photo search, voice control, our Best Self-Hosted AI Tools guide is the wider map. This one is what happens after you’ve already decided Ollama and Open WebUI are where you’re starting.

FAQ

What’s the difference between Ollama and Open WebUI?

Ollama runs the model itself and exposes an API. Open WebUI is the browser interface that talks to that API. You can use Ollama alone from a terminal, but Open WebUI is what makes it usable for anyone who isn’t typing commands, and it adds accounts, chat history, and file uploads on top.

Do I need a GPU to run a self-hosted LLM?

No, not for smaller models. An 8GB CPU-only machine handles 1B-3B parameter models without a GPU, just slower than you’re used to from a hosted chatbot. A GPU becomes worth adding once you want 7B-or-larger models to feel instant instead of merely usable.

Is Open WebUI free to use?

Yes, free to self-host, use, and modify. Its license changed from BSD-3-Clause to a modified license starting with version 0.6.6 that requires keeping Open WebUI’s branding visible unless you meet specific exemption criteria on its GitHub page, worth knowing if license purity matters to you, but it doesn’t change what running it costs: nothing.

Can I use Ollama without Open WebUI?

Yes, Ollama works entirely from the command line or through its API on its own. Open WebUI is an optional layer on top for anyone who’d rather use a browser than a terminal, most people end up adding it within the first day anyway.

Can I reach my self-hosted LLM from my phone or outside my home?

Yes, through Open WebUI’s web interface on your local network immediately, and from outside your home once you’ve put a reverse proxy or a mesh VPN like Tailscale in front of it. Don’t forward port 11434 itself, that’s Ollama’s unauthenticated API, not the login-protected interface.

Can Open WebUI connect to cloud models too, not just Ollama?

Yes. Alongside Ollama, Open WebUI can connect to OpenAI, Anthropic, any OpenAI-compatible API, and local runtimes like llama.cpp or vLLM, all from the same interface. Plenty of people run a local model by default and keep a cloud model connected for the occasional task that needs it.

Is it legal to self-host an LLM?

Yes. Ollama, Open WebUI, and the open models they run are legitimate open-source or source-available software, no different legally from installing any other program on hardware you own.

Ready to go beyond a single chatbot? Our Best Self-Hosted AI Tools roundup covers n8n for turning this same model into an agent, plus Immich and Home Assistant’s own local AI features. The Self-Hosted AI hub and Homelab hub round up everything else in this silo as we publish it, and the Docker & Containers hub covers the container basics this entire guide leans on.