Quickstart

From zero to a running OpenAI-compatible endpoint in ~5 minutes.

Tailor the examples to your setup:Modeldeepseek-ai/DeepSeek-V3.2moonshotai/Kimi-K2.6GPUAll GPUsMI300XMI325XMI355X

Prerequisites

• GPU — AMD Instinct (ROCm 6.3+)
• Docker 27+
• Python 3.11+
• A Hugging Face account (to pull model weights)

1. Install LLMBoost Hub

pip

pip install llmboost_hub
lbh --version

uv

uv is a fast Python installer.

uv tool install llmboost_hub                 # installs `lbh` on PATH
lbh --version
uvx --from llmboost_hub lbh --version         # or run one-off, without installing

2. Authenticate

hf auth login   # or: export HF_TOKEN=...

A free trial license is provisioned automatically on first serve — no extra step.

3. Serve a model

lbh serve deepseek-ai/DeepSeek-V3.2

That pulls the image, downloads the model if needed, and starts an OpenAI-compatible server on http://localhost:8000.

4. Send a request

OpenAI API

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
resp = client.chat.completions.create(
  model="deepseek-ai/DeepSeek-V3.2",
  messages=[{"role": "user", "content": "Say hello in one word."}],
)
print(resp.choices[0].message.content)

cURL

curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
  "model": "deepseek-ai/DeepSeek-V3.2",
  "messages": [{"role": "user", "content": "Say hello in one word."}]
}'

Already on OpenAI? You just changed the base URL — everything else in your code stays the same.

Next

• Run the server yourself (plain docker run)? → Manual
• Full LLMBoost Hub workflow (prep, run)? → LLMBoost Hub
• Curious how much faster? → Benchmarks