Ahmad Mujahid/Blog
← Blog

Install Local LLM in Ubuntu Laptop

Ahmad Mujahid Abdurrahman

Install Local LLM in Ubuntu Laptop

After getting a new MacBook as my main laptop, my Lenovo Slim 7 Pro X, which had been my daily driver for the past three years, was basically left sitting on the edge of my desk collecting dust.

Lenovo Slim 7 Pro X laptop My Lenovo sitting collecting dust

Rather than letting an otherwise perfectly usable laptop go to waste, I decided to give it a new job: running local LLMs as an AI server for my MacBook.

I installed Ollama and Open WebUI on the Lenovo, running Ubuntu 22.04 LTS, and configured it so I could access the models from my main laptop. I also had to tweak the firewall configuration to allow the Docker container to communicate with the host machine.

Here's how I set everything up.

Local AI Setup Manual: Ollama + Open WebUI on Ubuntu

This guide outlines how to install Ollama and Open WebUI on Ubuntu, including the necessary firewall configurations to allow the Docker container to talk to the local host machine.

Step 1: Install Ollama & Download Models

  1. Install Ollama via the official installation script:

    curl -fsSL https://ollama.com/install.sh | sh

  2. Download your preferred models to the local system:

    ollama pull deepseek-r1:latest
ollama pull llama3.2:latest

    you can replace deepseek-r1 and llama3.2 with any other models you want to use.

  3. Verify the models are loaded successfully:

    curl http://localhost:11434/api/tags

Step 2: Install Docker Engine

  1. Update package indices and install the core Docker runtime components:
    sudo apt update && sudo apt install -y curl docker.io
  2. Enable the Docker service to automatically start on boot:
    sudo systemctl start docker
sudo systemctl enable docker
  3. Configure non-root user permissions so you can run Docker commands without prefixing sudo:
    sudo usermod -aG docker $USER
    Note: You must log out and back into your Ubuntu user session for these group changes to apply.

Step 3: Configure Ubuntu UFW Firewall (The Fix)

By default, Ubuntu’s Uncomplicated Firewall (ufw) blocks communication passing between Docker's virtual network interface bridges and local system ports. You must explicitly allow the internal Docker bridge adapter (docker0) to connect to Ollama's port (11434).

  1. Permit Docker bridge traffic to access the local Ollama API server:
    sudo ufw allow in on docker0 to any port 11434 proto tcp
  2. Reload or ensure UFW is active:
    sudo ufw reload

Step 4: Deploy Open WebUI

Deploy Open WebUI using Docker. The deployment configuration includes the --add-host flag to resolve the host system's IP address dynamically, and injects the OLLAMA_BASE_URL environment flag.

Option A: Standard Docker Run Command

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway   -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Option B: Docker Compose Setup (docker-compose.yml)

Create a file named docker-compose.yml and add the following content:

version: '3.8'

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
    volumes:
      - open-webui:/app/backend/data
    restart: always

volumes:
  open-webui:

Launch the configuration by executing:

docker compose up -d

Step 5: Initial Web Verification

  1. Access the web control layout by navigating your browser to: http://localhost:3000
  2. Create your administrator account (the first account registered acts as the system owner).
  3. If models are missing from the top selector menu on initial load:
    • Navigate to Settings > Connections > Ollama API.
    • Confirm the URL is explicitly targeting: http://host.docker.internal:11434
    • Click the Refresh (circular arrows) icon to verify the connection turns green.

Step 6: Accessing the Ollama API from Your MacBook

On your MacBook, open a terminal and test the connection to the Ollama API running on your Lenovo:

curl http://<Lenovo_IP>:11434/api/tags

Replace <Lenovo_IP> with the actual IP address of your Lenovo laptop on the local network.

Step 7: Accessing the LLM WebUI from Your MacBook

Open a web browser on your MacBook and navigate to:

http://<Lenovo_IP>:3000

Again, replace <Lenovo_IP> with the actual IP address of your Lenovo laptop.