How to Self-Host Ollama on a VPS

Ollama downloads and runs large language models such as Llama and Gemma on your own server, with a command-line tool and a REST API on port 11434.

This guide installs Ollama by hand on Ubuntu 24.04 with its official Linux install script, which runs it as a systemd service, and shows the official Docker image as an alternative. VeloxaHost servers have no GPU, so models run on the CPU.

Prefer one click? Deploy Ollama from the VeloxaHost marketplace and skip the manual steps; this guide is for when you want to run every command yourself.

What you need

• A server running Ubuntu 24.04 LTS (pick it as the image when you create the server) with root access. Use SSH or the browser terminal on the server's page.
• Enough memory for your model. A model has to fit in RAM along with its context, and the disk has to hold every model you pull. The Ollama library lists llama3.2:1b at 1.3 GB, llama3.2:3b at 2.0 GB and llama3.1:8b at 4.9 GB.
• A domain and a dedicated IPv4 address only if you want to reach the API over HTTPS from elsewhere.

For 1B to 3B models, a Veloxa 2 (2 vCPU, 4 GB RAM, 80 GB disk) at $15 a month, billed at $0.03 an hour and capped at the monthly price, is enough. For 7B and 8B models such as llama3.1:8b, use a Veloxa 4 (4 vCPU, 8 GB RAM, 160 GB disk) at $30 a month, billed at $0.05 an hour and capped at the monthly price.

Install Ollama

1. Run the official install script. It installs Ollama, creates an ollama user, and sets up and starts the ollama systemd service.

   curl -fsSL https://ollama.com/install.sh | sh
   

2. Check the service and the version.

   systemctl status ollama
   ollama -v
   

3. Read the service logs if anything looks wrong.

   journalctl -e -u ollama
   

Alternative: the Docker image. If you prefer a container, install Docker Engine from Docker's apt repository, then start Ollama's CPU-only image:

# Add Docker's official GPG key:
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

docker run -d -v ollama:/root/.ollama -p 127.0.0.1:11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama3.2

Ollama's docs publish the port as -p 11434:11434, which listens on every interface. Docker's docs warn that published ports bypass ufw, so the 127.0.0.1: prefix above keeps the API private. Models live in the ollama volume. The rest of this guide uses the systemd install; with Docker, prefix ollama commands with docker exec -it ollama.

First login

Ollama has no accounts or web interface. Your first login is pulling a model and calling the API. Pull a small model, try it, and see what is installed and loaded:

ollama pull llama3.2:3b
ollama run llama3.2:3b "Explain DNS in two sentences."
ollama ls
ollama ps

In ollama ps, the PROCESSOR column shows 100% CPU on VeloxaHost, which is expected. Now call the REST API. Responses stream by default; "stream": false returns one complete answer:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

The API also offers /api/chat and an OpenAI-compatible endpoint under /v1.

Then adjust the server. Ollama reads environment variables from its systemd unit. Run sudo systemctl edit ollama and add lines under [Service]:

[Service]
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_CONTEXT_LENGTH=8192"

sudo systemctl daemon-reload
sudo systemctl restart ollama

By default a model stays in memory for 5 minutes after use, and the context window is 4096 tokens. A longer keep-alive avoids reloading between requests; a larger context costs more RAM. Models are stored in /usr/share/ollama/.ollama/models. To move them, set OLLAMA_MODELS the same way and give the ollama user the directory with sudo chown -R ollama:ollama <directory>.

Secure it

Ollama binds to 127.0.0.1:11434 by default, and the API docs say local requests need no API key. Anyone who can reach the port can use your models, so keep it on loopback.

Private use: an SSH tunnel. Nothing is exposed. From your computer, using the first port of your server's block (shown under "How to connect" on the server's page), which forwards to SSH:

ssh -p 20000 -L 11434:127.0.0.1:11434 root@203.0.113.10

While it is open, tools on your computer can use http://localhost:11434.

Do not forward a port of your block straight to 11434. Forwarded traffic arrives on the server's network interface, so that would mean setting OLLAMA_HOST=0.0.0.0:11434 and publishing an API with no password.

Shared access: Caddy with a password. For other machines or a team, add a dedicated IPv4 address (a paid add-on under "Network access"; the console shows the current price). Add it to the interface and to the network configuration so it survives a reboot, and point an A record at it:

ip addr add 198.51.100.5/32 dev eth0

Install Caddy from its apt repository and hash a password. Caddy only accepts hashed passwords, and its docs recommend argon2id:

sudo apt install --yes debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo chmod o+r /usr/share/keyrings/caddy-stable-archive-keyring.gpg
sudo chmod o+r /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy

caddy hash-password --algorithm argon2id

Replace /etc/caddy/Caddyfile with this, using your domain and the hash. Ollama stays on loopback, and the Host header matches the value in Ollama's own proxy example:

ollama.example.com {
	basic_auth argon2id {
		apiuser <paste the hash here>
	}
	reverse_proxy localhost:11434 {
		header_up Host localhost:11434
	}
}

sudo systemctl reload caddy
curl -u apiuser:yourpassword https://ollama.example.com/api/generate -d '{"model": "llama3.2:3b", "prompt": "Hello", "stream": false}'

Caddy gets a Let's Encrypt certificate on its own once the A record points at the server and ports 80 and 443 are open. Another route is a DNS-challenge certificate served on a forwarded port of your shared block.

Firewall. VeloxaHost firewalls are network-level and stateful. Until you attach one, all traffic is allowed (except outbound TCP 25). Under Firewalls, New Firewall, Add Rule, allow TCP 22 only from YOUR.IP/32, and TCP 80 and 443 from 0.0.0.0/0 only if you run Caddy. Attach it on the server's page.

Back it up

Models can be pulled again from the Ollama library, so they do not need a backup of their own. Back up what you made: Modelfiles for custom models, your systemd override and your Caddyfile.

mkdir -p /root/ollama-backup
ollama show --modelfile mymodel > /root/ollama-backup/mymodel.Modelfile
cp /etc/systemd/system/ollama.service.d/override.conf /etc/caddy/Caddyfile /root/ollama-backup/

To recreate a custom model, run ollama create mymodel -f ./mymodel.Modelfile. If you signed in to ollama.com, your Ollama key is in /usr/share/ollama/.ollama.

VeloxaHost snapshots capture the whole server, models included, so a restore brings everything back without downloads. Snapshots are instant, free (5 on Starter, 30 on Pay as you go) and stored in the same region. On the server's page, turn on automatic snapshots (hourly to weekly, kept 1 to 30 days). On the Backups page, schedule backups (every 6 hours to weekly, keeping 1 to 10) and download one now and then for a copy off the server.

Update it

Take a snapshot first: server page, Snapshots, enter a name, Create. Then run the install script again, as Ollama's docs describe, and check the version:

curl -fsSL https://ollama.com/install.sh | sh
ollama -v

To install a specific version, set OLLAMA_VERSION to a release number from the GitHub releases page (replace X.Y.Z):

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=X.Y.Z sh

With Docker, pull the newer ollama/ollama image, remove the container and start it again with the same docker run command; the ollama volume keeps your models.

To remove Ollama completely, follow the uninstall steps in Ollama's Linux docs:

sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm -r $(which ollama | tr 'bin' 'lib')
sudo rm $(which ollama)
sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollama

Troubleshooting

Answers are slow. Ollama is running on the CPU. Use a smaller model such as llama3.2:1b and keep the context length modest.

A model will not load, or memory runs out. The model and its context must fit in RAM. Memory grows with OLLAMA_NUM_PARALLEL times OLLAMA_CONTEXT_LENGTH, and on CPU up to 3 models can be loaded at once. Unload one with ollama stop <model>, pick a smaller model, or move to a larger size.

Another machine gets "connection refused". Ollama listens only on 127.0.0.1. Use the SSH tunnel or Caddy. If Caddy returns errors from Ollama, check the header_up Host line.

The API returns 503. The server is overloaded and its queue (OLLAMA_MAX_QUEUE, 512 by default) is full.

Logs. Follow them with journalctl -u ollama --no-pager --follow --pager-end, or docker logs ollama for the container.

FAQ

Which VeloxaHost size fits Ollama?

For 7B and 8B models, a Veloxa 4: 4 vCPU, 8 GB RAM and 160 GB of disk at $30 a month, billed at $0.05 an hour and capped at the monthly price. For 1B to 3B models, a Veloxa 2 (2 vCPU, 4 GB RAM, 80 GB disk) at $15 a month, billed at $0.03 an hour, is enough.

What license is Ollama under?

Ollama is released under the MIT license. The models you pull have their own licenses, set by whoever published them, so check each model's terms before using it in a product.

Does Ollama get a GPU on VeloxaHost?

No. VeloxaHost servers have no GPUs, so Ollama runs models on the CPU. Expect much slower generation than on a GPU; it suits small models, batch jobs and internal tools.

Does Ollama send my prompts to ollama.com?

Not for local models. Ollama's FAQ says that when you run locally, Ollama does not see your prompts or data. Cloud models are processed by ollama.com, and setting OLLAMA_NO_CLOUD=1 turns those features off.