Features
Everything you need to run LLMs in production — and nothing you have to babysit.
OpenAI-compatible API
Drop-in /v1. Keep your existing OpenAI client code.
Learn more →Integrations
OpenWebUI and agentic coding tools — OpenCode, Codex.
Learn more →Multi-GPU
Near-linear tensor/data parallelism across one node.
Learn more →Streaming & structured
SSE streaming, JSON-schema outputs, tool calls, vision.
Learn more →Benchmarks vs vLLM
Live throughput tables, plus scaling charts.
Learn more →Under the hood
- Auto-tuned — parallelism, batching, and memory tuned to your exact GPU. No knob-twiddling to get strong numbers.
- AMD Instinct™ — best-in-class on AMD Instinct™ MI300X / MI325X / MI355X (ROCm).
- Model-agnostic — Llama, Qwen, Mistral, and more, from ~1B to 500B+ parameters, including vision models.
- Yours — runs on your GPUs; weights and prompts never leave your infrastructure.
New here? Start with the Quickstart, then dive into the guide for whichever capability you need.