open-webui on Debian 12

von breit

This is an open-webui example for Debian 12 (XEN VM)

running on Xen Hypervisor with:

  • CPU: AMD Ryzen 9 3900 (12 cores, 24 threads)
  • GPU: none
  • RAM: 128GB – DDR4 ECC
  • HDD: 2x SSD U.2 NVMe 1,92 TB (RAID 1)
  • Uplink: 1 Gbit

vm config:

  • CPU: 12 cores
  • RAM: 84GB
  • HDD: 300GB (LVM)
  • OS: naked Debian 12

Setup within the VM:

apt update && apt install htop net-tools mc rsync screen nload locate bash-completion python3.11 pip iptraf nginx nginx-common snapd -y
snap install core
snap install --classic certbot
ln -s /snap/bin/certbot /usr/bin/certbot

Now edit your vhost config in:

/etc/nginx/sites-available/ollama.yourdomain.cfg:

server {
    
    server_name ollama.yourdomain.xy;
    
    location / {
        proxy_pass http://localhost:8080;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_buffering off;
        client_max_body_size 50M;
        proxy_read_timeout 10m;
    }
    
    listen 80;
}

and a symlink to

ln -s /etc/nginx/sites-available/ollama.yourdomain.cfg /etc/nginx/sites-enabled/

Now check your nginx config with:

nginx -t

if ok, restart nginx with

service nginx restart

now run

certbot

for SSL in your nginx vhost. Now install ollama with

curl -fsSL https://ollama.com/install.sh | sh

If this done you can pull modells from

https://ollama.com/search

Example:

ollama pull codellama:13b

if done, install open-webui with:

pip3.11 install open-webui --break-system-packages

Once that is done, you can now start the opwn-webui. This can be done with

open-webui serve

I run the whole thing within a screen instance. Thats all! If everything has been done correctly should now see in the browser:

https://ollama.yourdomain.xy

You should now be able to access the page.

Here a funny example in german: