Attention

In one line

A way for a model to decide which earlier words matter most for understanding the current one.

Picture it

Reading 'the trophy didn't fit in the suitcase because it was too big' and knowing 'it' points to the trophy.

Under the hood

Each token forms queries, keys and values; weights come from query and key similarity. Multi-head attention runs many of these in parallel.