Learn
Tokens
Plain-language explainer of Tokens, with related catalog pages.
Definition
Tokens are the pieces of text a model reads and writes — often subwords rather than whole words.
How it works
A tokenizer splits input into token IDs. Context windows, output limits, and API bills are usually measured in tokens, not characters.