In plain words
A transformer is a neural-network architecture built around attention mechanisms and other learned transformations. It is widely used for language and has been adapted to images, audio, and other data.
A closer look
Attention lets a token’s representation incorporate information from other relevant positions. Multiple attention heads can capture different relationships, while feed-forward layers transform the resulting representations. Positional information helps account for order. Stacking these components produces increasingly rich transformations of the input.
Transformers come in encoder-only, decoder-only, and encoder–decoder forms. Many generative language models use decoder-only designs with causal attention, so predictions cannot look ahead to future tokens. Architecture details vary, and efficiency techniques can change how attention is computed.
In practice
In “The keys are on the table because I left them there,” attention can help connect “them” with the relevant earlier words. This is a mathematical interaction between representations, not literal human attention.
A useful distinction
A transformer is an architecture, not a particular chatbot or a synonym for all AI. Not every AI model is a transformer, and using the architecture does not by itself establish any particular capability.