Configuration reference
LLMBoost runs great with defaults. When you need to tune, here's the surface.
llmboost serve --help always lists the complete, version-correct set of flags
for your build. The tables below cover the ones you'll reach for most.
vLLM-compatible CLI
Migrating from vLLM? llmboost serve exposes a vLLM-compatible serve
interface — the same flag names and semantics — so your existing launch
scripts, configs, and tooling run against it unchanged. (Same idea as the
OpenAI-compatible API: meet you where your stack
already is.)
llmboost serve <Repo/Model-Name> [flags]
| Flag | Default | What it does |
|---|---|---|
--port N | 8000 | Port for the HTTP server. |
--tensor-parallel-size N | auto | Shard the model across N GPUs. |
--max-model-len N | model max | Cap context length (prompt + output). |
--gpu-memory-utilization 0–1 | tuned | Fraction of VRAM for weights + KV cache. |
--max-num-seqs N | tuned | Max concurrent sequences per batch. |
--enforce-eager | off | Skip graph capture (faster start, lower peak throughput). |
--scheduling-policy fcfs | fcfs | Request ordering policy. |
--disable-llmboost-opts | off | Run without LLMBoost's licensed optimizations (diagnostics). |
For more detail, use llmboost --help (or, per command, llmboost <command> --help);
llmboost serve --help always lists the complete, version-correct flag set.
Environment variables
| Variable | Purpose |
|---|---|
HF_TOKEN | Hugging Face token for gated/private model downloads. |
Other advanced scheduler knobs are documented in llmboost serve --help. (Server
log verbosity is fixed for now and not user-configurable.)
LLMBoost Hub config
LLMBoost Hub (lbh) manages images and models:
lbh list # models + status
lbh serve <Repo/Model> -- <serve flags> # flags after `--` pass to the server
Licensing
LLMBoost activates a license on first start (a free trial is provisioned automatically) and then serves locally. Keep the host online for first activation. For enterprise seats and offline licensing, contact mangoboost.io/contact.