Set Up Qwen 2.5 in Docker with Ollama on Azure VM - Complete Guide

Qwen 2.5 is a robust AI model built for advanced text processing and natural language understanding. Deploying it within a Docker container on an Azure Virtual Machine (VM) provides a streamlined, scalable and isolated execution environment.

Before proceeding, make sure you have:

• An Azure VM (Ubuntu recommended) with Docker installed.
• SSH access to the instance.

Access your VM by executing the following command in your terminal:

ssh -i your-private-key.pem azure-user@your-vm-ip

Once connected, update the package lists:

sudo apt update && sudo apt upgrade -y

Launch an Ollama container to manage the model execution:

docker run -d --name ollama -v ollama:/root/.ollama -p 11434:11434 ollama/ollama

After the container is up and running, gain shell access to it:

docker exec -it ollama /bin/bash

Retrieve the Qwen 2.5 model inside the Ollama container:

ollama pull qwen:2.5

This command ensures all dependencies are properly installed.

Start the model with the following command:

ollama run qwen:2.5

Test it by entering a prompt:

>>> Describe the history of artificial intelligence.

To interact with Qwen 2.5 through a browser, deploy the Open WebUI tool:

docker run -d -p 3000:8080 -e 
OLLAMA_BASE_URL=http://<YOUR-VM-IP>:11434 -v
open-webui:/app/backend/data --name open-webui --restart always
ghcr.io/open-webui/open-webui:main

Now, open http://<YOUR-VM-IP>:3000 in a browser to start using Qwen 2.5 via a web interface.

By running Qwen 2.5 on an Azure Virtual Machine with Docker and Ollama, you achieve an efficient and scalable AI deployment. This approach simplifies model execution while maintaining an isolated and reusable setup for various workloads.

Ready to elevate your business with cutting edge AI and ML solutions? Contact us today to harness the power of our expert technology services and drive innovation.