developers
Inference
Inference for builders: related frameworks, APIs, and models in the catalog.
Definition
Inference is running a trained model to produce outputs — chat replies, embeddings, images, or tool calls.
How it works
Latency, cost, batching, hardware, and quantization all sit in the inference stack. Training happens once; inference is the ongoing bill.