learn
Inference
What Inference means, how it works, and which tools and models on this site relate to it.
Definition
Inference is running a trained model to produce outputs — chat replies, embeddings, images, or tool calls.
How it works
Latency, cost, batching, hardware, and quantization all sit in the inference stack. Training happens once; inference is the ongoing bill.