Skip to main content

LLMBoost metrics

Show information regarding the LLM inference requests

Metric nameTypeDescription
num_active_requestsgaugeNumber of ongoing chat/completion requests in the node
num_total_requests_totalcounterTotal number of requests served by the LLMBoost engine