Ollama

Ollama is what most people mean when they say they’re running an LLM locally. It’s a command-line tool and a background service in one: give it a model name and it downloads the weights, picks a quantization, and serves that model over a local API on port 11434. No account, no API key, nothing leaving your network unless you go looking for its optional cloud tier. This page is a quick reference: what Ollama is, its real license, a one-line Docker install, and the model families it runs. For hardware sizing and the full Compose stack, see our complete Ollama + Open WebUI setup guide.

What is Ollama?

Under the hood, Ollama wraps llama.cpp, the inference engine Georgi Gerganov started that made running quantized language models on ordinary hardware practical. What Ollama adds on top is everything llama.cpp leaves for you to sort out yourself: a searchable model library, one-command downloads, automatic quantization picks, a persistent server process, and an API modeled closely on OpenAI’s, so many existing tools work with it almost unchanged. That’s most of why it became the default answer to “how do I run a model on my own hardware,” ahead of tools that are arguably more capable but fussier to set up. Worth being clear about: Ollama itself has no chat window, just a CLI and a daemon. Full source is on GitHub, past 16.8k forks and 229 releases.

Model families you can run

The model library turns over constantly, but the backbone is the open-weight families this whole space tracks: Meta’s Llama line (3, 3.1, 3.2, 3.3, 4), Google’s Gemma, Alibaba’s Qwen, Mistral and Mixtral, DeepSeek, and Z.ai’s GLM, alongside a rotating cast of newer names. Ollama’s own GitHub description name-drops “Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models,” a fair snapshot of how fast that list turns over. Scale is easy to underestimate: Llama 3.1 alone has been pulled over 116 million times through Ollama, and Mistral 7B past 31 million, just two entries among dozens of families and hundreds of tagged variants. Sizes on offer typically run from under a billion parameters to several hundred billion, so what actually runs well depends entirely on your hardware, not on Ollama.

Quick Docker install

For a production setup with Open WebUI, Compose, and reverse-proxy security, use the complete guide. To get Ollama itself running right now:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

That pulls the official ollama/ollama image from Docker Hub and exposes the API on port 11434. Add --gpus=all to that command for NVIDIA acceleration; for AMD, swap the tag to ollama/ollama:rocm and add the /dev/kfd and /dev/dri devices. Then pull and run a model:

docker exec -it ollama ollama run llama3.2

Ollama vs the alternatives

Ollama isn’t the only way to run models locally, and it isn’t always the right pick. LM Studio is the closest comparison: it wraps that same llama.cpp foundation in a desktop GUI, friendlier for anyone who’d rather not live in a terminal, but harder to run headless on a server, exactly where Ollama is strongest. Going a level lower, plain llama.cpp hands you more control over quantization and build flags, at the cost of doing by hand everything Ollama automates. GPT4All targets roughly the same audience as LM Studio, with its own bundled model picker and a bit less flexibility underneath.

	Ollama	LM Studio	llama.cpp
Interface	CLI + REST API	Desktop GUI	CLI / library
Headless server use	Built for it	Limited	Yes
Official Docker image	Yes	No	Community only
Best for	Servers, scripting, self-hosting	Desktop users who want a GUI	Maximum low-level control

• Official Docker image and a REST API that’s easy to script into other self-hosted tools
• One-command model pulls, no manual quantization decisions
• Runs headless on a NAS, mini PC, or home server, no desktop needed
• Free, MIT-licensed, works fully offline

• No graphical interface out of the box; you’ll want Open WebUI or similar
• Automatic quantization picks aren’t always what a power user would choose by hand
• The paid Cloud tier sits right alongside the free local tool, which can feel like upselling
• Multi-GPU setups need more manual tuning than the one-line install suggests

FAQ

Is Ollama made by Meta?

No. Ollama is built by Ollama Inc., a company with no connection to Meta. The name similarity to Llama, Meta’s model family, comes up often enough that Ollama’s own team has addressed it publicly.

Is Ollama free?

The software is free and MIT-licensed, nothing gated behind a paywall for local use. Ollama separately sells an optional Cloud plan for larger models on its own servers, a distinct product from the tool covered here.

Is it safe to run Ollama at home?

Running it locally carries the same basic precautions as any self-hosted service: keep it on your local network or behind a VPN, since the API has no authentication by default. The security section of our full setup guide covers locking it down before exposing it anywhere.

Do I need a GPU?

No. Ollama runs fine on CPU only for smaller models, just slower per response. A GPU mainly buys speed and headroom for bigger models, not a requirement to get started.

What’s the difference between Ollama and Open WebUI?

Ollama is the engine: it downloads and serves models over an API. Open WebUI is a separate, optional interface that talks to that API so you get a browser-based chat screen instead of a terminal prompt. Most people running Ollama long-term end up pairing the two, which is exactly what our complete setup guide installs.

For the wider picture of what’s realistic to self-host in this space, see our roundup of best self-hosted AI tools for a homelab, or browse everything else AI-related on the Self-Hosted AI category page.