Deploying OpenThinker 7B on Azure Using Virtual Machines: A Step by Step Guide

Deploying OpenThinker 7B on Microsoft Azure using Virtual Machines (VMs) is a straightforward approach that offers full control over the infrastructure. We will:

• Deploy an Azure Virtual Machine (VM)

• Install Docker on the VM

• Pull and run the OpenThinker 7B model as a Docker containerConfigure networking for external access

• Full control over resources - Customizable VM configurations
• Easier troubleshooting and debugging
• Cheaper for small scale deployments

Before starting, ensure you have:

• An active Azure subscription

• Azure CLI installed

• Docker installed on your local machine

• A pre built Docker image of OpenThinker 7B

Log in to Azure CLI

az login

If you have multiple subscriptions, set the active subscription:

az account set --subscription "your-subscription-id"

Create a Resource Group

az group create --name OpenThinkerRG --location eastus

This creates a resource group named OpenThinkerRG in the East US region.

Create an Azure Virtual Machine

az vm create \
  --resource-group OpenThinkerRG \
  --name OpenThinkerVM \
  --image UbuntuLTS \
  --size Standard_NC4as_T4_v3 \
  --admin-username azureuser \
  --generate-ssh-keys

This creates an Ubuntu VM with an NVIDIA T4 GPU (adjust VM size if needed).

SSH keys are automatically generated for secure access.

Connect to the VM via SSH

ssh azureuser@<public-ip-address>

Find your VM’s public IP using:

az vm list-ip-addresses --resource-group OpenThinkerRG --name OpenThinkerVM --output table

Update System and Install Docker

Once inside the VM, update and install Docker:

sudo apt update && sudo apt upgrade -y
sudo apt install -y docker.io

Enable Docker Service

sudo systemctl enable docker
sudo systemctl start docker

Verify Docker Installation

docker --version

Expected output:

Docker version <docker version>

Pull the Docker Image from Docker Hub or Azure Container Registry (ACR)

If the model image is on Docker Hub:

docker pull your-dockerhub-username/openthinker-7b:latest

If the image is in Azure Container Registry (ACR):

Login to ACR

az acr login --name OpenThinkerRegistry

Retrieve the ACR Login Server

az acr show --name OpenThinkerRegistry --query loginServer --output tsv

Pull the image

docker pull <acr-login-server>/openthinker-7b:latest

Run OpenThinker 7B Container

docker run -d --name openthinker-7b -p 80:11434 <acr-login-server>/openthinker-7b:latest

• -d runs the container in detached mode.
• -p 80:11434 maps port 80 of the VM to the model's port 11434.

Allow Traffic on Port 80

By default, Azure VMs block external traffic. Open port 80:

az vm open-port --port 80 --resource-group OpenThinkerRG --name OpenThinkerVM

Check Running Containers

docker ps

Expected output:

CONTAINER ID   IMAGE                	PORTS              	STATUS
[Container ID]   openthinker-7b:latest	0.0.0.0:80->11434/tcp   Up X minutes

Find the Public IP of the VM

az vm list-ip-addresses --resource-group OpenThinkerRG --name OpenThinkerVM --output table

Test API Response

Use cURL or a web browser to check if the model is running:

curl http://<public-ip>

Expected output:

{"message": "Model is up and running"}

Ensure the container starts automatically after a reboot:

Create a Systemd Service

sudo nano /etc/systemd/system/openthinker.service

Paste the following:

[Unit]
Description=OpenThinker 7B AI Model
After=network.target
[Service]
ExecStart=/usr/bin/docker start -a openthinker-7b
ExecStop=/usr/bin/docker stop openthinker-7b
Restart=always
User=root
[Install]
WantedBy=multi-user.target

Enable the Service

sudo systemctl daemon-reload
sudo systemctl enable openthinker.service
sudo systemctl start openthinker.service

If you need more computing power, consider:

• Upgrading to a more powerful VM (e.g., Standard_NC8as_T4_v3)

• Using multiple VMs with a Load Balancer

• Running multiple containers on the same VM

To run multiple containers:

docker run -d --name openthinker-7b-2 -p 81:11434 <acr-login-server>/openthinker-7b:latest

This runs a second instance on port 81.

To stop the VM:

az vm stop --resource-group OpenThinkerRG --name OpenThinkerVM

To delete the VM:

az vm delete --resource-group OpenThinkerRG --name OpenThinkerVM --yes

To delete the resource group (removes all related resources):

az group delete --name OpenThinkerRG --yes --no-wait

Deploying OpenThinker 7B on an Azure Virtual Machine provides a flexible and controlled environment. By using Docker, we can quickly set up and run the model, exposing it over the internet with minimal configuration.

Ready to transform your business with our technology solutions? Contact Us  today to Leverage Our AI/ML Expertise.