OpenAI-compatible API

LLMBoost serves the OpenAI REST API on /v1. If your app already talks to OpenAI, migration is a one-line base-URL change — no SDK swap, no rewrite.

Tailor the examples to your setup:Modeldeepseek-ai/DeepSeek-V3.2moonshotai/Kimi-K2.6GPUAll GPUsMI300XMI325XMI355X

Migrate in one line

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",  # ← point at LLMBoost
    api_key="not-needed",                  # local serving needs no key
)

TypeScript

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:8000/v1', // ← point at LLMBoost
  apiKey: 'not-needed',
});

cURL

export OPENAI_BASE_URL=http://localhost:8000/v1
export OPENAI_API_KEY=not-needed

Everything else — chat.completions.create(...), streaming, function calling — stays exactly as you wrote it.

Endpoints

Endpoint	Purpose
POST /v1/chat/completions	Chat (messages in, assistant message out)
POST /v1/completions	Raw text completion
POST /v1/embeddings	Embeddings (embedding models)
POST /v1/responses	OpenAI Responses API
POST /v1/audio/transcriptions · …/translations	Speech-to-text (audio models)
GET /v1/models	List served models
GET /health · GET /metrics	Liveness · Prometheus metrics

Chat completion

Python

resp = client.chat.completions.create(
  model="deepseek-ai/DeepSeek-V3.2",
  messages=[{"role": "user", "content": "Say hello in one word."}],
)
print(resp.choices[0].message.content)

TypeScript

const resp = await client.chat.completions.create({
model: 'deepseek-ai/DeepSeek-V3.2',
messages: [{role: 'user', content: 'Say hello in one word.'}],
});
console.log(resp.choices[0].message.content);

cURL

curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
  "model": "deepseek-ai/DeepSeek-V3.2",
  "messages": [{"role": "user", "content": "Say hello in one word."}]
}'

Compatibility notes

• Standard sampling params pass through: temperature, top_p, max_tokens, n, stop, seed, logprobs, …
• Streaming ("stream": true), structured outputs, and tool calls are supported — see Streaming, structured output & tools.
• Drop the Authorization header for local serving, or put LLMBoost behind your own gateway for auth.