What the stack needs from the machine
Two processes share the box. Open WebUI is a Python web application that stores users, chats and settings in SQLite inside its volume. The engine loads the model weights onto the card and does the inference. The front end is light; the engine sets the hardware requirement.
| Component | Figure | Scope |
|---|---|---|
Open WebUI :slim container, idle | about 300 MiB RAM | Project's own test: v0.12.3, Linux/arm64, Docker Desktop, no users or chats. Not a minimum. |
Image download, :main | 1.62 GB | Compressed, Linux/amd64, checked 16 October 2026 |
Image download, :slim | 197 MB | Same check; leaves out local embedding, speech and document tools |
| Model weights, 7-8B at 4-bit | 5 GB of card memory | Weights only, from our GPU catalogue's table |
| Model weights, 30-34B at 4-bit | 19 GB | Weights only |
| Model weights, 70B at 4-bit | 38 GB | Weights only |
Weights are a floor. The runtime and the key-value cache share the same card memory and the cache grows with context, so the catalogue tells you to add a quarter to a half on top. LLM hosting works through the arithmetic for each model size.
Picking the image tag
:main(same as:latest): the standard rolling image, with local embedding and speech-to-text models inside.:cuda: the standard image with NVIDIA support (CUDA 12.8). It moves Open WebUI's own embedding, reranking and Whisper models to the GPU; run it with--gpus all.:ollama: Open WebUI and Ollama in one container. With--gpus=allthe models run on the card, and Ollama's port 11434 stays inside the container unless you publish it.:slim: chat front end only, for when the engine and any document or voice services live elsewhere.:vX.Y.Z: pinned to one release and never changes. The docs recommend pinning for production.
For one GPU box serving a team, :ollama is the shortest path. If you already run vLLM or llama.cpp's server on the host, use :main or :slim and add the engine as a connection instead.
Install with the bundled engine
Prerequisites: the NVIDIA driver, Docker, and the NVIDIA Container Toolkit, registered with nvidia-ctk runtime configure --runtime=docker and a Docker restart. Then:
umask 077
printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > /root/open-webui.env
docker run -d -p 127.0.0.1:3000:8080 --gpus=all \
-v ollama:/root/.ollama \
-v open-webui:/app/backend/data \
--env-file /root/open-webui.env \
--name open-webui --restart always \
ghcr.io/open-webui/open-webui:ollamaThe docs pass the secret as -e WEBUI_SECRET_KEY=.... That works, but the value then sits in your shell history, so this version stores it in a root-owned file with mode 600. Keep the same key across updates: a new one logs every user out and breaks tokens encrypted with the old key.
Publishing -p 3000:8080 without an address binds every interface, and Docker's own rules route around ufw. Binding to 127.0.0.1 keeps the UI off the internet until a proxy is in front of it.
Claim the admin account privately
- From your workstation, tunnel the port:
ssh -L 3000:127.0.0.1:3000 root@your-server. - Open
http://localhost:3000and wait for Create Admin Account;docker logs -f open-webuishowsApplication startup completewhen it is ready. - Register. That account manages users and every instance-wide setting, and creating it turns New Sign Ups off.
- Pull a model: type its name into the model selector of a new chat and confirm the download.
- Only now put the proxy in front.
To let colleagues register, turn New Sign Ups back on under Settings > Admin > Authentication. New accounts sit as Pending until you approve them, unless Default User Role is set to user. The environment variables ENABLE_SIGNUP and DEFAULT_USER_ROLE only seed these values: once saved, the setting in the database wins over the variable. Turn off Community Sharing under Settings > Admin > General if nothing should leave the instance.
TLS in front, and connecting other engines
Caddy obtains the certificate itself, and its reverse proxy carries WebSocket connections with no extra directive; Open WebUI needs WebSockets for streamed replies. Open 80 and 443 in the firewall and keep 3000 closed.
# /etc/caddy/Caddyfile
chat.example.com {
reverse_proxy 127.0.0.1:3000
}The troubleshooting guide asks for CORS_ALLOW_ORIGIN set to every address users open the UI by, such as https://chat.example.com; leaving it wrong shows up as WebSocket errors. For an OpenAI-compatible engine, add a connection under Settings > Admin > Connections: URL http://host.docker.internal:8000/v1 for vLLM's default port, with /v1 at the end. That name resolves only if the container was started with --add-host=host.docker.internal:host-gateway, and the engine must listen on an address the Docker bridge reaches. llama-server defaults to 127.0.0.1, so it needs --host; keep the engine's port closed to the internet but allow it from the Docker bridge (for example ufw allow from 172.17.0.0/16 to any port 8000 proto tcp), and give it a key (--api-key on llama-server). Ollama on its own on the host needs OLLAMA_HOST=0.0.0.0 for the same reason, and the same bridge-only rule on port 11434.
Updates and backups
Everything (database, uploads, generated files) is in the open-webui volume; the bundled engine's models are in ollama. Back up before every update, since database migrations are one-way and rolling the image back does not undo them.
docker run --rm -v open-webui:/data -v /root/backup:/backup \
alpine tar czf /backup/openwebui-$(date +%Y%m%d).tar.gz /data
docker pull ghcr.io/open-webui/open-webui:ollama
docker rm -f open-webui
# then rerun the docker run command above, same env fileRead the release notes first, and clear the browser cache afterwards. Stopping the container before the tar run gives a quieter copy of the SQLite file; that is our advice, not the project's.
Running it on a whole GPU machine
A GPU dedicated server gives the engine a physical card passed through to your own OS: no vGPU profile, no MIG partition, no time-slicing, and nothing upgrades the driver or CUDA release under you. You also get every core, all the memory, the disks, root, and the server's iLO or iDRAC console on a separate management network, which is what you reach for when a driver change takes the network down. The port is unmetered, so model downloads and attachments are not counted.
- GeForce cards: no ECC and no certified driver; for experiments, where a flipped bit costs a retry.
- Workstation cards: certified drivers and ECC on most of the line.
- Compute accelerators: large memory and ECC, for serving a model to real traffic.
- Delivery: about 30 minutes after approval when the machine is already racked; 4-24 hours when it is built to order.
Wrong fit: if the chat only fronts a hosted API, the :slim image needs no GPU at all. If a model needs more memory than one card holds (96 GB is the largest we fit), that is a multi-GPU build, quoted by hand.
Questions
Does Open WebUI itself need a GPU?
No. It ships no models of its own for chat; the engine behind it does the inference. The :cuda tag only moves Open WebUI's local embedding, reranking and speech-to-text models onto the card. Put the GPU where the engine runs.
Which port does it use?
The container listens on 8080. The documentation maps it to 3000 on the host with -p 3000:8080; change the left number if 3000 is taken. Behind a proxy, bind that host port to 127.0.0.1 so the proxy is the sole way in.
Can I use vLLM or llama.cpp instead of Ollama?
Yes. Both expose an OpenAI-compatible API. Add it under Settings > Admin > Connections with the URL ending in /v1, using host.docker.internal from inside the container. For llama.cpp, set the connection's Provider to llama.cpp under Advanced to get the loaded-model indicator and the admin Eject button.
Who can register once it is public?
Nobody, until you allow it. Creating the first account turns New Sign Ups off. When you turn it on, new accounts wait as Pending for approval unless Default User Role is user. Larger teams can use OAuth or OIDC single sign-on instead.
I lost the admin password. Now what?
The project documents a reset: generate a bcrypt hash with htpasswd, then write it into the auth table of webui.db in the volume using a throwaway Alpine container. Emails are stored lower-case, so match the address in lower case.
Do I lose chats when I update?
Not if the container mounts the same volume and gets the same WEBUI_SECRET_KEY. Without a fixed key a recreated container generates a new one, which logs everyone out. Take a backup first anyway: a release with a database migration cannot be rolled back by changing the tag.