Run with LLMBoost Hub

lbh is the fastest path: one CLI that manages images, model assets, the license, and the server — on a single node or a Kubernetes cluster.

Tailor the examples to your setup:Modeldeepseek-ai/DeepSeek-V3.2moonshotai/Kimi-K2.6GPUAll GPUsMI300XMI325XMI355X

Install

pip install llmboost_hub        # or: uv tool install llmboost_hub
lbh --version
# upgrade later: pip install --upgrade llmboost_hub

Log in to Hugging Face (for weights) and your container registry:

hf auth login                  # or set HF_TOKEN
docker login -u <username>     # your MangoBoost-provisioned registry account

Registry access is provisioned by MangoBoost: email contact@mangoboost.io to sign up — include your Docker Hub username — and we'll invite that account to the LLMBoost image registry. Then docker login with it.

The one-liner to begin serving your model

lbh serve deepseek-ai/DeepSeek-V3.2    # pull + download + serve

Use the full Hugging Face name (Repo/Model-Name). lbh serve downloads the image and weights if they aren't present, then serves.

Verify it's serving — send a chat request and print the reply:

lbh test deepseek-ai/DeepSeek-V3.2

Command reference

lbh -h summarizes all commands; lbh <command> -h details one; lbh -v <command> adds verbose diagnostics.

Core

Command	What it does
lbh login	Import + validate the license file (prompts for the EULA if needed).
lbh fetch [model]	Refresh the supported-model lookup cache, filtered to your GPU.
lbh list [query] [--discover PATH]	List prepared models; --discover scans a dir for existing model folders.
lbh prep <model> [--only-verify] [--fresh]	Pull the image + download model assets; caches the resolved path.
lbh run <model> [opts] -- [docker flags]	Start a detached container (mounts workspace; maps GPUs).
lbh serve <model> [opts] -- [llmboost args]	Start the server inside the container.
lbh test <model> [--query "…"] [-t N]	Send a request to /v1/chat/completions.
lbh attach <model>	Open an interactive shell in the running container.
lbh stop <model>	Stop the running container.
lbh status [model]	Concise status for prepared models.
lbh completions	Print/install shell completions.

Advanced usage

Pass LLMBoost args through lbh serve

Anything after -- goes straight to the llmboost server inside the container — so every serve flag is available via LLMBoost Hub:

lbh serve deepseek-ai/DeepSeek-V3.2 --port 8011 -- \
  --tensor-parallel-size 4 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.9

lbh serve options (before --): --host, --port, --detached (don't wait for readiness), --force (skip GPU-utilization checks). The serve flags themselves are in the Configuration reference.

Pass Docker flags through lbh run

Likewise, flags after -- on lbh run go to docker run — for extra mounts, env, or device options:

lbh run deepseek-ai/DeepSeek-V3.2 -- -v /data:/data -e MY_VAR=1

Configuration

LLMBoost Hub resolves settings in order: environment variable → $LBH_HOME/config.yaml → built-in default.

Variable	Default	Purpose
LBH_HOME	~/.llmboost_hub	Root data dir (config, cache, models, license).
LBH_MODELS	$LBH_HOME/models	Downloaded model assets.
LBH_MODEL_PATHS	$LBH_HOME/model_paths.yaml	Saved model-name → host-path map.
LBH_LICENSE_PATH	$LBH_HOME/license.skm	License file.
LBH_AUTO_PREP	True	Auto-run prep when run/serve need missing assets.
HF_TOKEN	unset	Hugging Face token, forwarded into containers.

Prefer a plain docker run?

You can run the same server without LLMBoost Hub — see Manual.