2. Install Ollama on Ubuntu server

To install Ollama on your Ubuntu server, especially if you're aiming to integrate it with Open WebUI and leverage GPU acceleration, here’s a complete step-by-step guide based on the latest best practices 1:

✅ Prerequisites

• Ubuntu 22.04 or later (Ubuntu 24.04 is also supported)
• NVIDIA GPU with at least 16 GB VRAM (for GPU acceleration)
• Root or sudo access
• Stable internet connection

🧱 Step-by-Step Installation

1. Install NVIDIA Drivers (if using GPU)

sudo apt update
sudo apt install -y ubuntu-drivers-common
ubuntu-drivers devices
sudo ubuntu-drivers autoinstall
sudo reboot
nvidia-smi  # Confirm GPU is detected

2. Install CUDA Toolkit (Optional but recommended for GPU)

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda-repo-ubuntu2204-12-2-local_12.2.2-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-12-2-local_12.2.2-1_amd64.deb

sudo cp /var/cuda-repo-ubuntu2204-12-2-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt update
sudo apt install -y cuda
nvcc --version  # Verify CUDA installation

3. Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama  # Verify service is running

4. Test Ollama

ollama run tinyllama

🌐 Optional: Install Open WebUI (Web Interface for Ollama)

5. Install Docker & Docker Compose

sudo apt remove -y docker docker-engine docker.io containerd runc
sudo apt update
sudo apt install -y ca-certificates curl gnupg lsb-release

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker
docker compose version  # Verify

6. Clone and Run Open WebUI

git clone https://github.com/open-webui/open-webui.git
cd open-webui
docker compose up -d

Access WebUI at: http://<server-ip>:3000

7. Connect Open WebUI to Ollama

In WebUI:

• Go to Settings → Backends → Ollama
• Set API URL:
  • Local: http://host.docker.internal:11434
  • Remote: http://<server-ip>:11434