What is LLMBoost?
LLMBoost is MangoBoost's enterprise LLM inference server. Point your existing OpenAI client at it and get up to the throughput of vLLM on the same GPUs — no code changes, no data leaving your infrastructure.
One line to serve a model
lbh serve deepseek-ai/DeepSeek-V3.2
Why teams pick it
| Benefit | What it means |
|---|---|
| Faster | up to vLLM throughput on the same GPU. Benchmarks → |
| Drop-in | OpenAI-compatible HTTP API. Swap the base URL, keep your code. |
| Yours | Runs on your AMD Instinct GPUs. Your weights, your prompts, your cluster. |
| Hands-off | Auto-tunes parallelism, batching, and memory for your exact hardware. |
Who it's for
- Platform & infra teams running a shared inference endpoint — get more tokens per GPU (lower cost per token) without changing the apps that call it.
- Application teams already on the OpenAI API — re-point the base URL at LLMBoost and keep shipping; nothing else changes.
- On-prem & regulated environments that must keep models and data in-house — LLMBoost serves entirely on your own GPUs.
Get going
- Quickstart — first model serving in ~5 minutes.
- Run LLMBoost — the LLMBoost Hub (
lbh) one-liner, or a plaindocker run. - Benchmarks — the numbers vs vLLM, and how we measured them.
- Features — what's in the box.
- Troubleshooting — quick fixes for common issues.
Powered by MangoBoost's patent-pending auto-tuning, scheduling, and memory optimizations. Record-setting MLPerf Inference v5.0, v5.1, and v6.0 results on AMD Instinct™ GPUs.