Transformer

The transformer is the neural architecture behind most modern language and multimodal models. It uses attention to relate tokens in a sequence.

Each token attends to others, building contextual representations in stacked layers. Variants trade depth, width, and attention design for cost and quality.