Transformers

Self-attention is attention applied within a single sequence, letting every word gather context from every other word in the same text.

Multi-head attention runs several attention computations in parallel, each learning to focus on a different kind of relationship.

Positional encoding adds word-order information to a transformer, because attention alone treats input as an unordered set.

The KV cache stores past tokens' keys and values during text generation so each new token is produced without recomputing the whole sequence.

A transformer is the neural network architecture behind modern AI, using attention to process all words at once and learn how they relate.

Attention lets each word decide which other words matter most, so a model can understand context.